-- Logs begin at Thu 2022-06-30 02:17:39 UTC, end at Thu 2022-06-30 04:36:37 UTC. -- Jun 30 04:01:05.904012 dv-cinder-846842-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 04:01:05.912823 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 04:01:05 2022] *** Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: nodename: dv-cinder-846842-7-iscsi Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: machine: x86_64 Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: clock source: unix Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: pcre jit disabled Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: detected number of CPU cores: 4 Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: current working directory: / Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: detected binary path: /usr/bin/uwsgi-core Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: your processes number limit is 47680 Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: your memory page size is 4096 bytes Jun 30 04:01:05.914887 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: detected max file descriptor number: 2048 Jun 30 04:01:05.915612 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: lock engine: pthread robust mutexes Jun 30 04:01:05.915612 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: thunder lock: enabled Jun 30 04:01:05.915612 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 04:01:05.915612 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 04:01:05.962409 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: Python main interpreter initialized at 0x560759e829c0 Jun 30 04:01:05.962409 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: python threads support enabled Jun 30 04:01:05.962409 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: your server socket listen backlog is limited to 100 connections Jun 30 04:01:05.962409 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: your mercy for graceful operations on workers is 90 seconds Jun 30 04:01:05.962766 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 04:01:05.962766 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: *** Operational MODE: preforking *** Jun 30 04:01:05.963077 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: *** uWSGI is running in multiple interpreter mode *** Jun 30 04:01:05.963077 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: spawned uWSGI master process (pid: 80692) Jun 30 04:01:05.963291 dv-cinder-846842-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 04:01:05.964676 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: spawned uWSGI worker 1 (pid: 80702, cores: 1) Jun 30 04:01:05.964676 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: spawned uWSGI worker 2 (pid: 80703, cores: 1) Jun 30 04:01:05.964676 dv-cinder-846842-7-iscsi devstack@keystone.service[80692]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 04:01:11.328190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 04:01:11.328190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: from cryptography.utils import int_from_bytes Jun 30 04:01:11.329024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 04:01:11.329024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: from cryptography.utils import int_from_bytes Jun 30 04:01:11.329425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 04:01:11.329425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: from cryptography.utils import int_from_bytes Jun 30 04:01:11.329531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 04:01:11.329531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: from cryptography.utils import int_from_bytes Jun 30 04:01:11.377470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 04:01:11.377667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 04:01:11.377808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] command line args: [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 04:01:11.377951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 04:01:11.378071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 04:01:11.378153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 04:01:11.378293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ================================================================================ {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 04:01:11.378436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] command line args: [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 04:01:11.378519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] admin_token = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.378582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 04:01:11.378785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] config_dir = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.378909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ================================================================================ {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 04:01:11.379001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] admin_token = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] config_source = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] config_dir = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] debug = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] config_source = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.379915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] debug = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.380952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_config_append = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_dir = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_file = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_config_append = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.381965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_dir = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_file = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.382970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.383945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.384974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] publish_errors = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.385781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] publish_errors = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.386982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] standard_threads = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] standard_threads = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] use_journal = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.387961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] use_json = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] use_stderr = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] use_syslog = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] use_journal = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] use_json = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.388973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] use_stderr = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.388973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] use_syslog = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.389175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 04:01:11.389401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.389947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.390959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.391989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.392970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.393851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.394959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.395890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.396888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.397890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.398967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.399909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.400926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.401997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.402892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.403952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.404978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.405949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.406953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.407900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.408903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.409894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.410977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.411868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.412720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.external = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.413935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.password = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] auth.token = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.external = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.414823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.password = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] auth.token = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.415901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.416914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.417923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.418945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.419925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.420946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.421965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.422982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.423914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.424954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.425857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.426897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.427960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.428959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.429930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.430951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.431917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.432932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.433941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.434954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.435987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.436928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.437944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.438961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.439911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.440914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.441928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.442931 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.443931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.444956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] role.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] role.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.445858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.446949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] role.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] role.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.447985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.448996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.449889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.450905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.451937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.452952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.453963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.454969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.455923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.456937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.connection = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.457984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.458898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.459936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.connection = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.460968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.461971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80703) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 04:01:11.462082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.462230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.462470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.462704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.462947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.463201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.463451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.463681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.463920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.464153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 04:01:11.464322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80702) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 04:01:11.492092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.492368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.492532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.494369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.494605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.494828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.501033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.503143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.529406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.529553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.529893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.530135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.679805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.682147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.730729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.730913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.731878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.732129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.795492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.795690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.795783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.795937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.796884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.821675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.821946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.822960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.823153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.823153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.834061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.834835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.846674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.846881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.846995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.847863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.848080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.848166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80703) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.848246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80702) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 04:01:11.865654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.865868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 04:01:11.866166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.866292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.866450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 04:01:11.866535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 04:01:11.866633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.866718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.866884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 04:01:11.867007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.867887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.878251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.878526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.878649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.878768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.878850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.879052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.879209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.879424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.887382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.888005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.890650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.890933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.891236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.891359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.891489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.891592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.891791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.891918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.892852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.893846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.920073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.921329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.921691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.921965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.922924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.923202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.927260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.927741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.928147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.929333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.929609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.930057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.935826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.936109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.937277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.937556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.940603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.940890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.941179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.941470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.941748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.942141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.942805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.943126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.943658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.944283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.944566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.961284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.961563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.963799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.964072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.966190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.969511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.969859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.970155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.973152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.973441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.976715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.979215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.980956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.982790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:11.986488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.986612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.986742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.986826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.987049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.987365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.987651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.987934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.989338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.989624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.989919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:11.990246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.008180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.008463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.008730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.008985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.009497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.009787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.010056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.010319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.016837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.018264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.018535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.018805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.020099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.020375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.022543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.022827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.023122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.023407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.023681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.023824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.024050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.024329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.024613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.024875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.036936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.037681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.037968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.038249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.038372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.038512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.038605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.038870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.039148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.039459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.040193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.040491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.041152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.041452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.058952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.060305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.062210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.063589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.065529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.066912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.067277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.068974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.069465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.069739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.070014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.070600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.070882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.071204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.077937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.078568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.079565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.080209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.082777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.083068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.083465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.083591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.083854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.084322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.084680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.084965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.094104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.094227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.094358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.094460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.094655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.094780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.102622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80703) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.102905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80702) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 04:01:12.103206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.103927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.104950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.105985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.106112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.106237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.106362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80703) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.106483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.106608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80702) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 04:01:12.144045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560759e829c0 pid: 80703 (default app) Jun 30 04:01:12.144860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560759e829c0 pid: 80702 (default app) Jun 30 04:01:12.147521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce85c58-edd3-4794-825e-1899e57d4e80 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:12.147826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce85c58-edd3-4794-825e-1899e57d4e80 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:12.148111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce85c58-edd3-4794-825e-1899e57d4e80 None None] PATH_INFO: `/v3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:12.152384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Thu Jun 30 04:01:12 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:19.880674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:19.881235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:19.881235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:19.885577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:19.893242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:19.893696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:19.893813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:19.968994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: INFO dbcounter [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] Registered counter for database keystone Jun 30 04:01:19.994005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG oslo_db.sqlalchemy.engines [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80703) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 04:01:20.005960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writer thread running {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 04:01:20.199303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG passlib.handlers.bcrypt [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80703) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 04:01:20.199907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG passlib.handlers.bcrypt [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80703) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 04:01:20.264506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:20.267007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:20.398766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 507 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 04:01:20.590749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:20.660983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: INFO dbcounter [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] Registered counter for database keystone Jun 30 04:01:20.682835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG oslo_db.sqlalchemy.engines [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80702) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 04:01:20.694242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writer thread running {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 04:01:20.812888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455cc3-4741-4ea3-87dc-a1d1f50aadf7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:20.818424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:20.819206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:20.819494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:20.819772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:20.828970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:20.880815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING keystone.common.rbac_enforcer.enforcer [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 04:01:20.881386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8fa82-fe33-4236-8b8c-febc98d2ee1a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:20.891468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jun 30 04:01:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:01:20.898737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:20.923045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db948302-d204-4f82-bcd5-e2a551c915c7 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:20.931660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:20.933319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:20.933812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:20.934311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] PATH_INFO: `/v3/projects/447338b8480048d8be1a3dac45927add` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:20.939937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] RBAC: Authorizing `identity:get_project(project_id=447338b8480048d8be1a3dac45927add)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:20.991075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING keystone.common.rbac_enforcer.enforcer [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 04:01:20.991441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbc6c6-60dc-415f-89b6-895303ee6e40 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:20.996083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Thu Jun 30 04:01:20 2022] GET /identity/v3/projects/447338b8480048d8be1a3dac45927add => generated 344 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:22.976977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8233-f994-4b3b-8a28-afe7604df115 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:22.978061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8233-f994-4b3b-8a28-afe7604df115 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:22.978061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f8233-f994-4b3b-8a28-afe7604df115 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:22.984350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:22.990205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:22.990628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:22.990736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:23.010923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:23.012480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:23.020907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 30 04:01:23.043591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8233-f994-4b3b-8a28-afe7604df115 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:23.062473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f8233-f994-4b3b-8a28-afe7604df115 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:23.071863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:23.072792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:23.073171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:23.073547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:23.085805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:23.087651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe683c2f-7f3e-4921-86cf-41baacd65c70 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:23.096914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Thu Jun 30 04:01:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:01:23.105522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:23.118744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974ecfb-3a68-43b5-b6b3-ccaeb2e9175a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:23.124958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:23.125846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:23.126132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:23.126405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:23.127959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:23.129805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1881a57-52f3-44e7-aea2-5654d70854d4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:23.131497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Thu Jun 30 04:01:23 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:25.206285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa116-4dbc-4019-b4ba-8f3a3df95847 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.206867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa116-4dbc-4019-b4ba-8f3a3df95847 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.206867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08aa116-4dbc-4019-b4ba-8f3a3df95847 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.208252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:25.214365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.214847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.214935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.218477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7610f5-e360-4c7c-bdbe-8cc43cbab58f None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.218781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7610f5-e360-4c7c-bdbe-8cc43cbab58f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.219065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7610f5-e360-4c7c-bdbe-8cc43cbab58f None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.220691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:25.225279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.225571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.225849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.232968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:25.234217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:25.266044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG passlib.handlers.bcrypt [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80702) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 04:01:25.266469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG passlib.handlers.bcrypt [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80702) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 04:01:25.312180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:25.313784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:25.320171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:25.343333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.367549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:25.414509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:25.437061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.453053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ea73d-b32d-4a95-a4c9-a65dccae0be6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.487532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.488354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.488642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.488922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.500277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:25.511758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:25.512687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:25.513536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:25.518997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.537341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5bb34-3541-4af4-ba84-286c21df2ffa None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.539668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:25.575010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.575968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.576262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.576549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.603886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:25.622729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebb348-3c53-4adf-ba43-87b72b6c7ea5 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.645770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:25.647413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:25.657328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.658009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.658302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.658593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.659380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:25.659695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:25.661114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:25.673578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:25.679631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.749528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:01:25.755139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.758006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:25.780081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf3955-0f4b-42c6-b10a-8a6e76216c98 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.814293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.815212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.815501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.815777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.817022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:25.818582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:25.820443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:25.825563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.840408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:25.853399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc5f1b-10a1-4498-89f8-c0189f768e8f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.888677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.889490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.889772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.890053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.909793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:25.911322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:25.927343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b24514-c861-496e-8a75-f10a038980b0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:01:25.932329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:25.933136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:25.934110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:01:25.934110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:01:25.934110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:01:25.934110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:25.934110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:25.939494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.964489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:25.965288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:25.965573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:25.965851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:25.967045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:25.968545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:25.970141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:25.976177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:25.998279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:26.041324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a280f4-716f-4da1-a49b-99bec68a10b4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:26.056729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:26.075030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:26.075732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:26.076018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:26.076136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f213-df9f-47e7-a12f-cac2e84f6e75 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:26.076299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:26.077039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:26.077350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:26.078755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4723464-ee69-4623-b30d-7d99c1a49024 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:26.095974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:26.111900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:26.112594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:26.112883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:26.113173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:26.114059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.domain.id=default, user.id=d70cac4e4add4da9a183025769b64cc8) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:26.114375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:26.115988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbe372-c98d-42e0-aa00-aceb877f4ea4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:26.123874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jun 30 04:01:25 2022] GET /identity/v3/role_assignments?user.id=d70cac4e4add4da9a183025769b64cc8&scope.domain.id=default&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:28.417464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1d9f-103d-475f-9b3f-62ee35ced1ca None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.418012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1d9f-103d-475f-9b3f-62ee35ced1ca None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.418012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e1d9f-103d-475f-9b3f-62ee35ced1ca None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.419424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:28.425154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da76bf-4726-42bb-9205-9e1defc938fe None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.425573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da76bf-4726-42bb-9205-9e1defc938fe None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.425744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da76bf-4726-42bb-9205-9e1defc938fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.452154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-44da76bf-4726-42bb-9205-9e1defc938fe None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:28.453392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-44da76bf-4726-42bb-9205-9e1defc938fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:28.545469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:28.584699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e1d9f-103d-475f-9b3f-62ee35ced1ca None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:28.734433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ae72f-98e3-4e47-8841-b73901b7f9fa None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.734829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ae72f-98e3-4e47-8841-b73901b7f9fa None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.735002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ae72f-98e3-4e47-8841-b73901b7f9fa None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.737844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:28.740824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e1d9f-103d-475f-9b3f-62ee35ced1ca None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:28.742165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.742466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.742754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.767474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:28.783769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:28.785016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:28.800769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:28.801702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.801990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.802324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.830431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e5d7f749-267b-459c-88d8-31354313693b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:28.841741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:28.842616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:28.843415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 12/25] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:28.844365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2510c-6b49-4d4e-b49d-fda3b93f5680 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.844818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2510c-6b49-4d4e-b49d-fda3b93f5680 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.845238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2510c-6b49-4d4e-b49d-fda3b93f5680 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.846662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:28.852497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a7447-23e8-4256-aef7-54abb4616628 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:28.852958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a7447-23e8-4256-aef7-54abb4616628 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:28.853377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a7447-23e8-4256-aef7-54abb4616628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:28.881331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b11a7447-23e8-4256-aef7-54abb4616628 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:28.883836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b11a7447-23e8-4256-aef7-54abb4616628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:28.906534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 14/27] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:28.907146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:28.913320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:28.999599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 14/28] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:29.001777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95fabc-f532-44bd-a964-2593e61bd54d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.002259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95fabc-f532-44bd-a964-2593e61bd54d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.003549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95fabc-f532-44bd-a964-2593e61bd54d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.005428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 15/29] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:29.006940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95fabc-f532-44bd-a964-2593e61bd54d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.038508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432e98c-0a20-40da-a9e9-bac32266f6be None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.084227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.084914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.085208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.085484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.087280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:29.087381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.088772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b24b64-01ce-4ea7-a8d5-5cb9185719da None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.102227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:29.108109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.108411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.108700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.126455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:29.127719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:29.139409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95fabc-f532-44bd-a964-2593e61bd54d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.148092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:29.180363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.181791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.182261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.182718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.194140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.195912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.196904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:29.207235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 16/31] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:29.208220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:01:29.215836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:29.216711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:29.217500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:01:29.217500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:01:29.217500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:01:29.217500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:29.219397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:29.220367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.321413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c11b7-b517-42a0-8b57-49cb7da40adb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.340796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244eb1c-0297-4a0c-9661-d8c7ac63b7c8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.359756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.360449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.360744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.361024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.362002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.363906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.379912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.381689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.382187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.382792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.384570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.386269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.388334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 17/33] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:29 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:29.389372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.422280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:29.507374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44120577-a708-4aec-b21e-288f4fe7e76d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.526832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 17/34] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 30 04:01:29 2022] POST /identity/v3/roles => generated 219 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:29.528571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.549342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.550078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.550372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.550727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.551769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:29.552124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.554623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.569987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 18/35] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:29.572111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.618379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:29.672132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43efee-1410-463b-bb97-b16cc875fd43 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.705327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:29.725804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c195-969f-469f-9a91-2d106e790a55 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.768498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.771294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.771294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.771294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.771294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.772982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.774032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:29.775198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 19/36] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:29.775519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:29.776037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:29.776195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.776741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:29.777703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:29.779434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:29.814846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 18/37] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 30 04:01:29 2022] POST /identity/v3/roles => generated 225 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:29.816054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:29.929170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac7d3e-400c-4681-8364-4db904f565f1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:29.987059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7522bb-408c-4d9a-af9a-0e1fd0c1623c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:30.000437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:30.001506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.001985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.002452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.006301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:30.008309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96097f-9138-4187-ae15-00e5fa392205 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:30.009285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:30.038290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:30.039285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.039583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.040027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] PATH_INFO: `/v3/domains/default/users/d70cac4e4add4da9a183025769b64cc8/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.044765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d70cac4e4add4da9a183025769b64cc8, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:30.046613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917385de-c9c6-40dd-98a9-45b30ec4acb9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:30.074102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 19/38] 10.222.0.13 () {58 vars in 1361 bytes} [Thu Jun 30 04:01:29 2022] PUT /identity/v3/domains/default/users/d70cac4e4add4da9a183025769b64cc8/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:30.075660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1284e3-9f5f-4403-8cea-a13f397e3a04 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.075946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1284e3-9f5f-4403-8cea-a13f397e3a04 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.076228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1284e3-9f5f-4403-8cea-a13f397e3a04 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.078612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 20/39] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:30.085503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1c63f-a8cd-4c6a-a787-5e723998d7dc None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.085944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1c63f-a8cd-4c6a-a787-5e723998d7dc None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.086371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1c63f-a8cd-4c6a-a787-5e723998d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.116150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f0f1c63f-a8cd-4c6a-a787-5e723998d7dc None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:30.117890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f0f1c63f-a8cd-4c6a-a787-5e723998d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:30.184571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 20/40] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:01:29 2022] POST /identity/v3/projects => generated 301 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:30.212900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 21/41] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:30.220737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2ddf-f96c-4c5f-b9ac-df784bb26550 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.221414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2ddf-f96c-4c5f-b9ac-df784bb26550 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.221929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764f2ddf-f96c-4c5f-b9ac-df784bb26550 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.223828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 21/42] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:30.231006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.231391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.231622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.242252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f2ddf-f96c-4c5f-b9ac-df784bb26550 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:30.252835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:30.254205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:30.257457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:30.347389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 22/43] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:30.372219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:30.377896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764f2ddf-f96c-4c5f-b9ac-df784bb26550 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:30.413894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:30.423894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:30.424615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.424904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.425182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.425971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:30.428786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:30.449430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 22/44] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 30 04:01:30 2022] POST /identity/v3/roles => generated 223 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:30.620990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b7c4f-a780-49c0-9eb8-362f4cec5f2c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:30.658492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:30.659344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.659633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.659914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.661081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:30.663198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb816b8b-c241-4204-86ff-db06d2db8867 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:30.664923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 23/45] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:30.670587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:30.741771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:30.778551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d181554-b88a-48fa-abfc-af9983b22522 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:30.814831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:30.815535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:30.815835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:30.816107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:30.816824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:30.818317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f1ac4-eea5-41c4-b413-f8db407a5c53 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:30.847782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:01:30 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:32.229827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959ff80-ed13-4c38-9000-a8e9fc5935b5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.230377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959ff80-ed13-4c38-9000-a8e9fc5935b5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.230377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959ff80-ed13-4c38-9000-a8e9fc5935b5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.231940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:32.237314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586e91-b233-4fbd-8686-94409033843e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.237714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586e91-b233-4fbd-8686-94409033843e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.237854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da586e91-b233-4fbd-8686-94409033843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.256693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-da586e91-b233-4fbd-8686-94409033843e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:32.258220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-da586e91-b233-4fbd-8686-94409033843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:32.328913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:32.352252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959ff80-ed13-4c38-9000-a8e9fc5935b5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:32.461346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959ff80-ed13-4c38-9000-a8e9fc5935b5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:32.463730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:32.497805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:32.498539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.498821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.499131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.509860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:32.517646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:32.518480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:32.519328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:32.524778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da586e91-b233-4fbd-8686-94409033843e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:32.577259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:32.629535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da586e91-b233-4fbd-8686-94409033843e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:32.663571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:32.664254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.664542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.664822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.665578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:32.665888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:32.667324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:32.679561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 25/50] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:32.685245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:32.787684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275735d4-4aad-4f81-bb2d-3971245316a1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:32.822854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:32.823799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.824088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.824369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.825721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:32.827255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daa9888-b068-4f9d-8778-5e6aa87f4fb9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:32.828888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 26/51] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:32 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:32.834631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:32.855463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21202-a5c0-4d5a-b2ef-cc64c4b4a331 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.855757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21202-a5c0-4d5a-b2ef-cc64c4b4a331 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.856043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de21202-a5c0-4d5a-b2ef-cc64c4b4a331 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.857414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 27/52] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:32.862350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:32.862657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:32.862940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:32.880583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:32.881823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:32.908771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:32.935234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e9640a-c124-4c8d-9b56-4684a5b1decb None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:32.951926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 28/53] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:32.975686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:33.056220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:33.080705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:33.081516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:33.081821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:33.082106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:33.082292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef5dcc-0040-4252-b7e0-8c00a6f31578 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:33.083394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:33.084871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:33.086517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 26/54] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:33.092224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:33.117304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:33.118131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:33.118423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:33.118708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:33.120038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:33.121537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:33.123373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:33.128640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:33.159229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:33.194092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec5c29-5e8e-4fb0-b2e6-e66cf0ec760b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:33.228967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:33.229638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:33.229933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:33.230212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95af436-cee3-4a88-a75f-d7893ffecd49 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:33.230641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:33.231050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.domain.id=default, user.id=d70cac4e4add4da9a183025769b64cc8) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:33.231402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:33.232856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191221d-3208-43cb-8156-84e5f899db3e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:33.241568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 27/56] 10.222.0.13 () {56 vars in 1322 bytes} [Thu Jun 30 04:01:33 2022] GET /identity/v3/role_assignments?user.id=d70cac4e4add4da9a183025769b64cc8&scope.domain.id=default&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:33.269112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:33.269817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:33.270111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:33.270390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:33.271176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:33.272697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d8301-f2f0-49ed-a2a7-f82a83b8ef96 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:33.304068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 30/57] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jun 30 04:01:33 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:35.335743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6cc16-f103-483b-be11-1167a2204be8 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:35.336572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6cc16-f103-483b-be11-1167a2204be8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:35.336572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6cc16-f103-483b-be11-1167a2204be8 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:35.338806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:35.348213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:35.348617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:35.348872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:35.372919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:35.374132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:35.444243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:35.468490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6cc16-f103-483b-be11-1167a2204be8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:35.530577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:35.594605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6cc16-f103-483b-be11-1167a2204be8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:35.633094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:35.633896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:35.634184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:35.634472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:35.635750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:35.637248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db25650-f0aa-4b93-881a-f545d0bdf75a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:35.638909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:35.647365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:35.662453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:35.757795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c1161-f0a3-459f-bf83-68475e0977f4 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:35.804382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:35.805359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:35.805799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:35.806230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:35.807446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:35.809750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e82922-e398-4fdb-9f8c-8619d3fd9598 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:35.852161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:01:35 2022] POST /identity/v3/users => generated 275 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:39.053768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464522-7d8a-4a9a-93e3-1f94219a0fad None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.054825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464522-7d8a-4a9a-93e3-1f94219a0fad None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.055338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464522-7d8a-4a9a-93e3-1f94219a0fad None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.056924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:39.062273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bed0-517b-47a9-a6cd-bd44916e3af0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.063157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bed0-517b-47a9-a6cd-bd44916e3af0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.063687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912bed0-517b-47a9-a6cd-bd44916e3af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.088165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1912bed0-517b-47a9-a6cd-bd44916e3af0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:39.089722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1912bed0-517b-47a9-a6cd-bd44916e3af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:39.168509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.177228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 33/63] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:39.205586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81464522-7d8a-4a9a-93e3-1f94219a0fad None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.207245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2152d-54d3-4689-af52-8d5e310244b0 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.207550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2152d-54d3-4689-af52-8d5e310244b0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.207843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2152d-54d3-4689-af52-8d5e310244b0 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.215124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 34/64] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:39.215124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3e7b1-79d0-42c9-8be4-5b272e182f66 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.215124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3e7b1-79d0-42c9-8be4-5b272e182f66 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.215396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3e7b1-79d0-42c9-8be4-5b272e182f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.232659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c0b3e7b1-79d0-42c9-8be4-5b272e182f66 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:39.233888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c0b3e7b1-79d0-42c9-8be4-5b272e182f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:39.308686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 35/65] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:39.312512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad0a-e394-4c80-a454-29d737a02dd9 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.312891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad0a-e394-4c80-a454-29d737a02dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.313181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad0a-e394-4c80-a454-29d737a02dd9 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.314527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 36/66] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:39.321055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.321353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.321642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.347637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:39.348842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:39.362165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81464522-7d8a-4a9a-93e3-1f94219a0fad None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.376993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.408564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.410210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.410681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.411295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.424929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 37/67] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:39.425519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:39.426363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-eff3ee90-0c1d-419f-80ee-a7d3d3f1fac5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:39.435945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:39.437133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:39.438100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:39.438881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 31/68] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:39.440360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fccbc-9637-4374-9a61-459a7f4aa839 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.440851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fccbc-9637-4374-9a61-459a7f4aa839 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.441346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fccbc-9637-4374-9a61-459a7f4aa839 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.443023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 32/69] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:39.447731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fccbc-9637-4374-9a61-459a7f4aa839 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.501320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.546135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f299a073-a087-4e7d-ada3-1432ff98b174 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.558834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fccbc-9637-4374-9a61-459a7f4aa839 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.589443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.590276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.590569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.590855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.596637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.597456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.597747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.598027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.601924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:39.608429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:39.609745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:39.610581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 38/70] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:39.617192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-36797fc1-1c0d-4b06-9438-978d72790edf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:39.618139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:39.619313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 33/71] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:39.620114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.620797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.620797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.637756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:39.638972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:39.664127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.707981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 34/72] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:39.711976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.727188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf66d1b-69dd-4a74-8651-16ec57b3d72b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.744844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.761931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.762613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.762900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.763211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.763973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:39.764286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:39.765714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:39.773904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 39/73] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:39.773904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.816952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5409a-b67e-482d-aacf-1c7a2688ded6 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.850438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.851147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.851440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.851722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.852492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:39.852786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:39.854208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b0dfb-cb56-4766-b954-9b4ec5951aae None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:39.862285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 35/74] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:39.863519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445818-ef63-4d93-827c-64f84450711d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.863810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445818-ef63-4d93-827c-64f84450711d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.864061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01445818-ef63-4d93-827c-64f84450711d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.865345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 36/75] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:39.867848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01445818-ef63-4d93-827c-64f84450711d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.875476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b4697-3002-48ff-8033-c6f3627a5df1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:39.892015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.909208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:39.909891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:39.910181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:39.910467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:39.911257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:39.911634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:39.913030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:39.921032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 40/76] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:39.921817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:39.966447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:39.975705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01445818-ef63-4d93-827c-64f84450711d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.015447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.016366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.016657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.016937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.022416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bca18-e81a-4fd0-83b7-f004b64aca0f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.027564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:40.035680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:40.036672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:40.037450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 37/77] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:40.038081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.062896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.063983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.064278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.064563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.078172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.079632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34878c-c6de-4ab7-8bd0-013d0c334ccb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.081322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 41/78] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:39 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.084376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.084671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.084978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.100728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:40.101931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:40.122384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.141175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2701-8f52-4065-a059-6ed39c045ae3 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.169628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 42/79] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:40.170595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.186622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.187566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.187855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.188136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.189471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.190967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.197133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 38/80] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.197133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.202355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.272006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799cd52-5d8c-4e53-8aaf-d488be4b0e55 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.291862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709eb82-db40-4688-a500-7d2b1f2a144f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.306213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.307143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.307505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.307788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.308996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.310510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.312209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 43/81] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.312979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.326774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.327486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.327897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.328229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.329090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:40.329423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.330989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.339581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 39/82] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.340200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.354545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.415061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.419112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3956c5d-a54b-4f23-b9a2-c488b5c60d90 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.439790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716697-a3fb-427a-9d1c-396651600007 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.454259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.454906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.455221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.455502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.473199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.474062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.474419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.474701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.475935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.477472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.483617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 40/83] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.483617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.526407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.527941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.557770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.577537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121ebae-47c6-4630-92ed-8109995a877c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.613143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.613730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.614020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.614308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.628051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.632910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 44/84] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:40.633741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.685500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.686964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.735313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ec704-0a41-4299-a5c1-f4eb1c975c2b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.764395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.770534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.771606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.771606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.771938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.784286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 41/85] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:40.785122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.842021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:40.846854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.848397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.886392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387a1fd-741a-4b1b-be36-0f5a0c75c3b0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:40.920078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:40.920965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:40.921251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:40.921532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:40.922722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:40.924206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:40.925827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 42/86] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:40.928329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.947982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 45/87] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:40.948833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:40.982664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.032589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bb994-cc13-47e2-8963-473441d3f340 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.045909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.050026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd79ba-8dc4-4f39-ac9e-fd46113269f4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.066246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.066923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.067237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.067519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.068269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.069809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.086434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.087416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.087709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.087985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.089356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.090926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.092691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 46/88] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:40 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.093458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.102699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 43/89] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:01:40 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:41.102699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.210369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab592e98-c8cc-4650-b871-8b0f6864c189 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.225220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a76c9-7676-4a04-af67-89940112aef1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.228926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.249560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.250886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.251229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.251511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.253640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.255507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.257316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 47/90] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.257832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.265743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.266515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.267749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.267749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.294248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.356733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.358400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.382029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d207e-2f94-4126-b68b-596e6a512d8f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.421803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.422708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.422995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.423314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] PATH_INFO: `/v3/projects/8a8a9a2561444209be225a72fe610601` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.425054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] RBAC: Authorizing `identity:get_project(project_id=8a8a9a2561444209be225a72fe610601)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.426790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.429286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 48/91] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/projects/8a8a9a2561444209be225a72fe610601 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.430327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.493585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 44/92] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:41.494671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.504105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.561145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05661ef-cccf-49e5-9923-e5df732980ff None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.563354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.607148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.607840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.608181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.608661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.609938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:41.610391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.612301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.618664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e4244-d135-44e7-a72c-dc92437e7069 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.620248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 49/93] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=370601cf296e44ea95990467138eb5e5 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.621432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.684510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.686809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.688085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.688510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.689531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=d70cac4e4add4da9a183025769b64cc8) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:41.690743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.692500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.708194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 45/94] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/role_assignments?user.id=d70cac4e4add4da9a183025769b64cc8&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.709958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:41.813262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575649d-263f-49d8-9bdd-8fdf1bc721b8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.850818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.851903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.852384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.852854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=8a8a9a2561444209be225a72fe610601, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:41.854449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.856162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.862885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 50/95] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=8a8a9a2561444209be225a72fe610601&role.id=370601cf296e44ea95990467138eb5e5 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.863446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:41.912314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa588-57a0-4506-a993-18baeb0a0709 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:41.966288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:41.967726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:41.968214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:41.968701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:41.973516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:41.975739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede7052-88b9-4946-8372-46ee604cbfee None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:41.978004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 46/96] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:41.984128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:42.039963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:42.138346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b75f9-0549-4e1a-81d8-2fe893f4ea88 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:42.175558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:42.176243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:42.176533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:42.176816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:42.178123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:42.178445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:42.180046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ec32d7-44f7-44bd-9a85-57a74ce7e39e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:42.189805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 51/97] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:41 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:43.839112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714283b1-b029-4e10-9587-447a3693c8f7 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:43.839597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714283b1-b029-4e10-9587-447a3693c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:43.839597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714283b1-b029-4e10-9587-447a3693c8f7 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:43.840989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 47/98] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:43.846137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5484ed-520d-4b78-a193-2b894c090f06 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:43.846509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5484ed-520d-4b78-a193-2b894c090f06 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:43.846710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5484ed-520d-4b78-a193-2b894c090f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:43.867209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3f5484ed-520d-4b78-a193-2b894c090f06 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:43.868730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3f5484ed-520d-4b78-a193-2b894c090f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:43.956458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 52/99] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:43.991601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714283b1-b029-4e10-9587-447a3693c8f7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.073673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0f9d-4e70-4e41-9340-1b7cc214ecbe None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.074313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0f9d-4e70-4e41-9340-1b7cc214ecbe None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.074784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d0f9d-4e70-4e41-9340-1b7cc214ecbe None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.076809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 53/100] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:44.088658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac26b5-6afe-4363-b5c3-ae277d8a9c13 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.089322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac26b5-6afe-4363-b5c3-ae277d8a9c13 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.089793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac26b5-6afe-4363-b5c3-ae277d8a9c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.120163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-32ac26b5-6afe-4363-b5c3-ae277d8a9c13 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:44.121733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-32ac26b5-6afe-4363-b5c3-ae277d8a9c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:44.130570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714283b1-b029-4e10-9587-447a3693c8f7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:44.165690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:44.166511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.166802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.167085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.170213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:44.171967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:44.174083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:44.175976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 48/101] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:44.183006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.188107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:44.276621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 54/102] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:44.278319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3341477-56d8-438b-8d21-a759b128bed9 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.278810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3341477-56d8-438b-8d21-a759b128bed9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.279330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3341477-56d8-438b-8d21-a759b128bed9 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.280857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 55/103] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:44.326995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67474817-01be-4cca-97ee-9a278928c629 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:44.336678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.337227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.337705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.371189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:44.373533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:44.421011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:44.422054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.422536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.423507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.425111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:44.427336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:44.486469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 49/104] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:01:44 2022] POST /identity/v3/projects => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:01:44.487786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.501042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 56/105] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:44.536333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.615594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:44.687685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b59f17-4af6-402f-9bb3-121cdf232eae None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:44.741279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:44.742113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.742408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.742724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.747116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:44.756563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:44.764702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:44.765561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:44.766366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 50/106] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:44.772100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.790842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b64ec0-1310-48ff-95d8-73c95777d709 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:44.842617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:44.843727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.844024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.844717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.860059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-944310ec-4113-4d65-b90d-a05cb45c529a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:44.870994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:44.871884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:44.872668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:44.873245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 57/107] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:44.876843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ba12c-05e0-4492-ae82-e7d68b343a69 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.876843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ba12c-05e0-4492-ae82-e7d68b343a69 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.876843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ba12c-05e0-4492-ae82-e7d68b343a69 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.877493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 58/108] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:44.882141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ba12c-05e0-4492-ae82-e7d68b343a69 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:44.891328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a990e-4a2c-42fa-b5e9-80a5f6b1fed4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:44.911765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:44.946209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:44.947024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.947345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.947625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:44.948403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:44.948712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:44.950124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb43bd-1411-45ed-9ac5-94f0906095b0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:44.971001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 51/109] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:44.973749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:44.974419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:44.974662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.015035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:45.016284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:45.081334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ba12c-05e0-4492-ae82-e7d68b343a69 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.139973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 52/110] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:45.141739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.146989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.147845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.148131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.148410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.150111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:45.150430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.154934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.173556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 59/111] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:45.178418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.192426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.319313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843922fe-bae4-4eea-bda6-045e5aa9e7a0 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.322328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be80ef0-9c9c-40b5-bae6-76f66dd1b383 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.331081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.359649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.360626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.360859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.361140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.362414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.363835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.364027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.364773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.365621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 60/112] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:45.365977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.366268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] PATH_INFO: `/v3/users/89f68b5f957945ab9733376b4a928414` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.366529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.386664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=89f68b5f957945ab9733376b4a928414)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.388651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef053df-2ec9-4eab-a31e-1442e0dcb3d9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.391259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 53/113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/users/89f68b5f957945ab9733376b4a928414 => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:45.392834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098cd88-d374-4c08-bbae-bc2cc95babe8 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.393367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098cd88-d374-4c08-bbae-bc2cc95babe8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.393650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098cd88-d374-4c08-bbae-bc2cc95babe8 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.395868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 54/114] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:45.397307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098cd88-d374-4c08-bbae-bc2cc95babe8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.454863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.491932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4661ee-efdf-4168-9a05-519efe052966 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.523667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098cd88-d374-4c08-bbae-bc2cc95babe8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.530401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.531644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.531935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.532215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.543474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:45.551080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:45.551962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 61/115] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:45.555575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.563397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.564529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.564987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.565279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.566496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.568203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c6e8a-31ed-479a-9c7d-19a4382ec205 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.574720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 55/116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:45.575924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467b77f-dfb1-435f-8311-a99fd6861d7a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.576222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467b77f-dfb1-435f-8311-a99fd6861d7a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.576507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467b77f-dfb1-435f-8311-a99fd6861d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.596276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c467b77f-dfb1-435f-8311-a99fd6861d7a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:45.597626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c467b77f-dfb1-435f-8311-a99fd6861d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:45.624599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.665846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb79aa1-15e7-4df1-934a-854de9ff79f2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.667933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 56/117] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:45.669145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130fa2-b271-4afe-bcbb-de484f182d7e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.669450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130fa2-b271-4afe-bcbb-de484f182d7e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.669735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130fa2-b271-4afe-bcbb-de484f182d7e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.671169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 57/118] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:45.675594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130fa2-b271-4afe-bcbb-de484f182d7e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.710531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.711220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.711509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.711788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.715463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.781043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130fa2-b271-4afe-bcbb-de484f182d7e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:45.797098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.798625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439c7e-2be8-41e5-90d2-f800cb33e2fa None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.818666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:45.819398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.819690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.819972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.821040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:45.821469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:45.823617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:45.841196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 58/119] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:45.841196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:45.873788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.920942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 62/120] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:45.920942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:45.920942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:45.920942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:45.938559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:45.939818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:45.966716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:45.970625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e21dad-3841-443b-87c7-603528f8e2a8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.005915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.006845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.007166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.007456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/c461c48ade384f048bfd341c19ff0197/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.007612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 63/121] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:46.015323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.016600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, user_id=c461c48ade384f048bfd341c19ff0197, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.018087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.037989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 59/122] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:45 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/c461c48ade384f048bfd341c19ff0197/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:46.038854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.143322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc722c1-9b14-4650-8ff8-26d3689df748 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.180161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.196449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.197665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.198095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.198513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.200337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca264cf9-1da2-48d8-89a8-c5f3f71b1c08 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.217424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:46.225562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:46.226504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 64/123] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:46.228115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.251459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.252432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.252726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.253011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.254358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.256209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.258255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 60/124] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.259482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.264644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.356228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f3b65-be9a-4e35-ad63-d0ef91de7e1d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.392197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.393236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.393686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.394009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.395171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e667c62-7b8c-4bc8-a9ea-1c375e50bb5a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.395506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.397469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.399342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.400211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.440082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.441209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.441640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.442103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.457568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:46.466649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:46.467792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:46.468740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:46.470097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 61/126] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:46.475813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.493501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.518970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.533190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5203f5-e340-4042-8630-396c54c2881e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.580272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.581249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.581757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.582263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.583265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:46.583779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.585375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.599300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 66/127] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.600505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.604126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9314d95-7317-4cac-82e3-e7209bc88a69 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.650440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.651923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.652805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.653398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.655160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.657232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.659798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 62/128] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.661044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.709315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dceb88-cd79-40cf-a8ef-e134b395227e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.744452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.745235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.745604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.745956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.746876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=89f68b5f957945ab9733376b4a928414) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:46.747294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.749112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.755756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 67/129] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/role_assignments?user.id=89f68b5f957945ab9733376b4a928414&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.757077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.762913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.778539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff8e72-075b-4705-bc0a-5ec6b83dac33 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.797106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:46.819579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.820680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.821169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.821646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:46.822640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:46.823185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:46.824878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:46.834934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 63/130] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:46.836197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:46.952435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d620034-ed8d-4d1d-aa74-4ca84cbc7886 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.979228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae95ef-9534-4151-b9a5-00219e316d87 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:46.996190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:46.997871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:46.998169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:46.998601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/d70cac4e4add4da9a183025769b64cc8/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.001371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, user_id=d70cac4e4add4da9a183025769b64cc8, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.002852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.020558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 64/131] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:46 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/d70cac4e4add4da9a183025769b64cc8/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:47.021663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.023717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.025128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.026561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.027574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.027977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.029571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.031359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381e580-26da-486a-a4b6-a932a5e0fa03 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.035861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 68/132] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:46 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:47.039335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212829e-1d2d-403c-a62e-35629412b672 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.039678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212829e-1d2d-403c-a62e-35629412b672 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.040027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212829e-1d2d-403c-a62e-35629412b672 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.041593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 69/133] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:47.046858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212829e-1d2d-403c-a62e-35629412b672 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.157379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15852f5d-a491-4304-b80a-a9bf5879e924 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.174240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.197639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212829e-1d2d-403c-a62e-35629412b672 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.198825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.200176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.200716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.201190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.202857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.204624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397d1fc-88e3-43c5-b8d5-ed4e8f4ac6d1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.206749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 65/134] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:47 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:47.212338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.212868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.213346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.236458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.237326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.237609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.237892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] PATH_INFO: `/v3/projects/8a8a9a2561444209be225a72fe610601` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.239359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] RBAC: Authorizing `identity:get_project(project_id=8a8a9a2561444209be225a72fe610601)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.241094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.242999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 70/135] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:47 2022] GET /identity/v3/projects/8a8a9a2561444209be225a72fe610601 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:47.243641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.250003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:47.251864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:47.367034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3254e4-f7d0-4e48-b776-92da3758698c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.380584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 66/136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:47.382472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.385246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.424365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.426226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.428555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.429075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.431523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.433857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.435644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 71/137] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:47 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:47.443882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.498049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.521069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a3f90-29fc-4c6e-94bf-fe5c3b11738a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.563169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.566418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.566418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.566418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] PATH_INFO: `/v3/projects/8a8a9a2561444209be225a72fe610601/users/c461c48ade384f048bfd341c19ff0197/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.567079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] RBAC: Authorizing `identity:create_grant(project_id=8a8a9a2561444209be225a72fe610601, user_id=c461c48ade384f048bfd341c19ff0197, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.568847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.586330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 67/138] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:47 2022] PUT /identity/v3/projects/8a8a9a2561444209be225a72fe610601/users/c461c48ade384f048bfd341c19ff0197/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:47.587194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.663074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.705390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a3d1-ae99-4116-8248-6f66829f5c46 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.713967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af530bb-1abe-4912-b1df-2d8afd76d3ec None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.757181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.758120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.758416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.758701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/c461c48ade384f048bfd341c19ff0197/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.761910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, user_id=c461c48ade384f048bfd341c19ff0197, role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.763449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.781940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 68/139] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:47 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/c461c48ade384f048bfd341c19ff0197/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:47.799135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.800355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.800793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.801254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.804005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:47.805583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9475-d554-487f-81e6-a1c03eb5acb5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:47.808689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 72/140] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:47.813541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:47.956800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285bf52-de3c-4db8-a6a5-b4883f06a370 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:47.983122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:47.996520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:47.997362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:47.997519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:47.997798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:47.998549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:48.000219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe9b35-f3b3-441a-b600-d13e8cd001b0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:48.025664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 69/141] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:01:47 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:48.398184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de0d92-1e7c-4862-b6b7-88fb26800f16 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:48.398665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de0d92-1e7c-4862-b6b7-88fb26800f16 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:48.398665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de0d92-1e7c-4862-b6b7-88fb26800f16 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:48.400131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 73/142] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:48.405632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:48.405934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:48.406226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:48.432060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:48.433269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:48.512487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 70/143] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:48.551146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de0d92-1e7c-4862-b6b7-88fb26800f16 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:48.674788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:48.688344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de0d92-1e7c-4862-b6b7-88fb26800f16 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:48.728684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:48.729507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:48.729799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:48.730080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:48.743133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:48.751200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:48.752118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:48.752934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 74/144] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:48.758830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:48.870295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbaa4b-4b81-4708-b20d-fc470211eef8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:48.880540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:48.910082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:48.910782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:48.911071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:48.911830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:48.912618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:48.912981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:48.914429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:48.923970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 71/145] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:48.930343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:49.037391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe659b66-9520-4132-a42c-3a047a0351cd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:49.077250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:49.078198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.078485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.078763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.081310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:49.082794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cdf4f8-dc31-4643-9e08-679350cfbb75 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:49.084541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 75/146] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:48 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:49.090203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:49.204392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec40ce6-60fe-4de8-b76d-ea769738edec None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:49.240239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:49.240916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.241315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.241603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.283602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:49.316272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:49.317837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:49.379839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741841-f762-492c-8396-d1718278731f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.380264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741841-f762-492c-8396-d1718278731f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.380515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7741841-f762-492c-8396-d1718278731f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.381908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 76/147] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:49.397126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.397333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.397566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.422425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:49.423653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:49.461381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:49.466837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 72/148] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:49.486974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:49.574646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:49.588708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 77/149] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:49.611615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ff735-971f-4699-b06c-d3d7a31a1069 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:49.626337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:49.648468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:49.653307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.655245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.655245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.656159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:49.658509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:49.661679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 73/150] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:49 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:49.671142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:49.891351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd4fd-2067-45ed-b2f3-c90a5caf90c7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:49.921314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223fce0-308c-4f2c-ae3c-08d92ee8e87a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:49.992416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:49.993396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:49.993834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:49.994253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:49.995293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:49.995764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:49.997565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:50.010885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 74/151] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:49 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=370601cf296e44ea95990467138eb5e5 => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:50.014460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:50.015343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.015626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.015900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.044101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:50.052276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:50.053710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:50.054485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 78/152] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:50.060309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:50.070360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:50.273581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e61af-b409-4ea3-94e3-0e26d49998a1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:50.315153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:50.315850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.316135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.316414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.317182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:50.317567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:50.318929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:50.331149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 75/153] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:50.337731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:50.353258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:50.475356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ff64-bb01-4fa2-8eb1-ef78adc80f34 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:50.520303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:50.521248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.521531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.521807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] PATH_INFO: `/v3/users/89f68b5f957945ab9733376b4a928414` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.523081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] RBAC: Authorizing `identity:get_user(user_id=89f68b5f957945ab9733376b4a928414)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:50.524704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:50.526543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 79/154] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:50 2022] GET /identity/v3/users/89f68b5f957945ab9733376b4a928414 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:50.532235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:50.610188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:50.651767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455dc02a-f543-40ef-9a86-0b9ed3a3e101 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:50.689420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:50.690311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.690658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.690944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.692206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:50.693846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c639116-1434-4e18-9b4c-42a8df358182 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:50.695880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 76/155] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:50 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:50.701397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:50.736462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a2ceb-1538-4afb-be23-3b10be97e9af None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.736778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a2ceb-1538-4afb-be23-3b10be97e9af None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.737024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a2ceb-1538-4afb-be23-3b10be97e9af None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.738601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 77/156] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:50.749612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.749902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.750177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.776327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:50.777468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:50.845687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:50.902138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e11e6-9bcf-46c0-99fa-f17f7969f79e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:50.929036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 78/157] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:50.977191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:50.978120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:50.978404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:50.978684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/89f68b5f957945ab9733376b4a928414/roles/e8be34dd9fd146969ddeabb56c1172b6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:50.984438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, user_id=89f68b5f957945ab9733376b4a928414, role_id=e8be34dd9fd146969ddeabb56c1172b6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:50.985932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:51.007257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:51.012796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 80/158] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:50 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/users/89f68b5f957945ab9733376b4a928414/roles/e8be34dd9fd146969ddeabb56c1172b6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:51.109764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:51.164010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ffae7-b901-4374-99e6-43c80163e306 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:51.219134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:51.219134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.219134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.219134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.244047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:51.258515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:51.260180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:51.261252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 79/159] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:51.269254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:51.431644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83b1e6-10fb-4b5a-b9fa-ce8a3e314b0a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:51.466836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:51.467552 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.467842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.468126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.468896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:51.469207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:51.470604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116cf5c-fa2c-45e6-9384-67120965f401 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:51.476410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:51.480009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 81/160] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:51.486958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:51.668557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6eb9a-b2f3-49e0-bc18-565b151f0a0a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:51.737320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:51.807527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:51.808501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.808785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.809110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] PATH_INFO: `/v3/users/d70cac4e4add4da9a183025769b64cc8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.812852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e542-52a5-43b4-b742-a4d5c4598813 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.813148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e542-52a5-43b4-b742-a4d5c4598813 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.813436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e542-52a5-43b4-b742-a4d5c4598813 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.814846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 82/161] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:51.815708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] RBAC: Authorizing `identity:get_user(user_id=d70cac4e4add4da9a183025769b64cc8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:51.817220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:51.819077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 80/162] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:51 2022] GET /identity/v3/users/d70cac4e4add4da9a183025769b64cc8 => generated 247 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:51.823233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783223-d52e-468e-adc2-808e929d08d4 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.825557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783223-d52e-468e-adc2-808e929d08d4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.825876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4783223-d52e-468e-adc2-808e929d08d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.828024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:51.855986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b4783223-d52e-468e-adc2-808e929d08d4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:51.857207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b4783223-d52e-468e-adc2-808e929d08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:51.953714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 83/163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:51.957009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6f83-2652-4875-a5e4-b41e8e731e4c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.957303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6f83-2652-4875-a5e4-b41e8e731e4c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.957579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b6f83-2652-4875-a5e4-b41e8e731e4c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.963615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 84/164] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:51.966071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:51.966361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:51.966642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:51.990325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:51.992256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:52.107434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 85/165] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:52.108334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:52.134542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fed46e-d123-4a8b-9a74-4c2ab0786a06 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.169059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.169735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.170023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.170375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.237785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:52.243022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:52.246573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:52.260533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:52.398702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54edb9ee-cdbe-43ab-a8a9-206f893a3d1a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.409971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 81/166] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:52.410900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:52.471866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.473107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.473587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.474084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.492812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:52.509939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:52.510869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:52.511954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:52.513632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e48b8-ad6b-442c-bc5a-b9950ff025a2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.520339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 86/167] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:52.521671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:52.539515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:52.559226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.560065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.560345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.560618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.575947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:52.585381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:52.586318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:52.587146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 82/168] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:52.587856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:52.693848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fa3d-4b13-4538-8cf1-e98cc5cf1f32 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.729114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218387f-c747-4b93-9874-49297df3013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.729800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218387f-c747-4b93-9874-49297df3013b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.730088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218387f-c747-4b93-9874-49297df3013b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.730370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218387f-c747-4b93-9874-49297df3013b None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.731158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218387f-c747-4b93-9874-49297df3013b None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:52.731476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218387f-c747-4b93-9874-49297df3013b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:52.732884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218387f-c747-4b93-9874-49297df3013b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:52.742221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 83/169] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:52.743391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca147f8-d6aa-405f-91bb-147f6d32bc9f None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.743685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca147f8-d6aa-405f-91bb-147f6d32bc9f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.744070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca147f8-d6aa-405f-91bb-147f6d32bc9f None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.745308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 84/170] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:52.746004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca147f8-d6aa-405f-91bb-147f6d32bc9f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:52.747043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e1a36-59bc-445a-b89d-01d4f401a812 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.763748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:52.783121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:52.844446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.845228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.845519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.845800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.847017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:52.848544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78376644-3a6f-4efc-9ba5-1209a50deea6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:52.850452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 87/171] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:52.858818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.859133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.859430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.859991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca147f8-d6aa-405f-91bb-147f6d32bc9f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:52.889091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:52.890107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:52.903406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:52.904313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:52.904781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:52.905204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:52.906299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:52.907454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:52.910320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:52.926528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:52.929296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.020462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.039842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 88/173] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:53.044048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.066305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a122e-1d97-4c88-97d3-8d36b5d568fb None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.106463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.107415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.107697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.107969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.109173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.110796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.112531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 86/174] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:52 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:53.113782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.180989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3f1c9-7f25-49d0-be1b-cd6fd41c3457 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.198676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.217888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.218557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.218758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3c415-bfda-4a41-8e97-7a8e60b5a586 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.218846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.219124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.219981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=d70cac4e4add4da9a183025769b64cc8) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:53.220297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.221852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572047e4-bab2-4ebe-ac62-9ffe0cf9cca5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.228286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 89/175] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/role_assignments?user.id=d70cac4e4add4da9a183025769b64cc8&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:53.229328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e609f-9ea1-45d9-ac64-a5ab10c41423 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.229614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e609f-9ea1-45d9-ac64-a5ab10c41423 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.229883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e609f-9ea1-45d9-ac64-a5ab10c41423 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.231129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 90/176] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:53.235532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e609f-9ea1-45d9-ac64-a5ab10c41423 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.269468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.271024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.272554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.273480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] PATH_INFO: `/v3/users/c461c48ade384f048bfd341c19ff0197` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.275210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] RBAC: Authorizing `identity:get_user(user_id=c461c48ade384f048bfd341c19ff0197)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.277737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe63b1-eac6-457c-b0a1-fd38921630d9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.280229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 87/177] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/users/c461c48ade384f048bfd341c19ff0197 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:53.282301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c494d-5751-49a1-891b-b65863315d16 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.282571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c494d-5751-49a1-891b-b65863315d16 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.282849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c494d-5751-49a1-891b-b65863315d16 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.284281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 88/178] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:53.285225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c494d-5751-49a1-891b-b65863315d16 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.300357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.369031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e609f-9ea1-45d9-ac64-a5ab10c41423 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.400783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c494d-5751-49a1-891b-b65863315d16 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.408046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.408852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.409134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.409406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.421613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-6b534a23-2d52-40da-9b41-2b3ae3480c53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:53.430432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:53.431387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:53.432194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:53.433255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 91/179] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:53.434360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.434645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.434922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.450836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.451577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.451914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.452195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.454606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:53.455826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:53.496866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.529287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.530775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86446-6fd4-47a3-9be2-42fcc759f8a1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.538225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 92/180] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:53.539352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.551971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.656457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f7f707-ef4b-4803-b06c-65a7e3858f89 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.662708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 89/181] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:53.662708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.662708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.662708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.677125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:53.678656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:53.702383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.703055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.703380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.703664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.745237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.756570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 90/182] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:53.757724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.780807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.782271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.805372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:53.867967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab8dd0-4d22-4f0c-9d49-85ae6a010be7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:53.883269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 93/183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:53.888568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.904696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:53.905384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:53.905667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:53.905941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:53.906822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:53.907271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:53.909372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:53.917443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 91/184] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:53.918260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:53.966788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.001759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa45f54-f787-485c-b52a-b062be84979d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.034384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a1132-1239-4420-8c2c-1320048d71d2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.035944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.036805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.037444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.037767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.038081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.039348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.040827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.042611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 94/185] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.043940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.077296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.078189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.078472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.078747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] PATH_INFO: `/v3/projects/8a8a9a2561444209be225a72fe610601` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.079982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] RBAC: Authorizing `identity:get_project(project_id=8a8a9a2561444209be225a72fe610601)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.081582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.083448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 92/186] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:53 2022] GET /identity/v3/projects/8a8a9a2561444209be225a72fe610601 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.084219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.146782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f006e-beba-4a1e-b97c-29c871291acf None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.180466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.181284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.181569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.181846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.184934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15b123-246b-432c-ad22-7b702440431e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.188649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.193031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:54.201398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:54.202272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 95/187] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:54.203944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.220235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.221115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.221398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.221693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.222874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.224541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35622d41-4c2e-41bd-b901-992fdf1c7945 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.226246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 93/188] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.227432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76834-4c6e-48b1-8d65-b55b77c63f85 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.227716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76834-4c6e-48b1-8d65-b55b77c63f85 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.227991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c76834-4c6e-48b1-8d65-b55b77c63f85 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.229176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 94/189] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:54.229844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c76834-4c6e-48b1-8d65-b55b77c63f85 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.272428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.312086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc140e3-21de-455d-b346-c7a235b5920f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.333500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c76834-4c6e-48b1-8d65-b55b77c63f85 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.350166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.351069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.351366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.351659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.365367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.366841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.368545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 96/190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.368875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.369624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.369897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.370184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.370893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.371961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.372360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.393956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 95/191] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:01:54 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:54.394779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.431878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.478953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf707a-ae74-455e-b4fa-b34eafc99bf5 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.500705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.510404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee6d0b-398b-4094-b6c2-adc01e621964 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.516419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.517109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.517392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.517666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.518517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=8a8a9a2561444209be225a72fe610601, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:54.518830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.520359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.527270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 97/192] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=8a8a9a2561444209be225a72fe610601&role.id=370601cf296e44ea95990467138eb5e5 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.528023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.559180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.559747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.559747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.560070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.561883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:54.561883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.563694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60570e78-bf06-4f36-bcaa-8001451ad29d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.573953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 96/193] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.578709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.578709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.578709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.597913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:54.599210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:54.681843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5cbd9-75c6-4942-88c0-40b14a9d752b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.698344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 97/194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:54.700323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.707498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.724836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.725319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.725674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.726019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.726984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=c461c48ade384f048bfd341c19ff0197) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:54.727520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.729099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.736317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 98/195] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/role_assignments?user.id=c461c48ade384f048bfd341c19ff0197&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.738234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.835873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:54.860674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36494a1b-05a1-4ff9-b29f-73d28f746015 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.881396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27341d-8b8f-48eb-a738-67d42ab5b226 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:54.908661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.909617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.909903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.910648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.923167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:54.924084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:54.924367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:54.924643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:54.926046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:54.928763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:54.930929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 99/196] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:54.931960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:54.995869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.032628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.034130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.059720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0365327-039f-4f3f-a68f-f11927f1a9f8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.101931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.103132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.103637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.104107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.116440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:55.124585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:55.125487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:01:55.126276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:55.129593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 100/197] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:55.129880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.131540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.150380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 98/198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:55.155361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.268606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f852119-4d2a-4303-af3e-f96eccef68e7 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.269804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dad41-7ec7-4664-bca7-05b4c1bff576 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.287240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.308697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.309406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.309763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.310045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.310810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:55.311126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.312811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.313907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.314898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.315477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.315760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.323572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 99/199] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:55.324458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.385771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.419688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.421174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.459800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd94f1a7-5736-4ed4-9c5c-cdf22faaabe8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.505464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.506974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.507771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.508283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.510437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.512847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.514812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:55.515868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.534250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 101/201] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:55.539406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.555714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.647730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8c97f2-4cfc-4db4-8999-ee52d42684ab None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.658330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4d6e4-e416-4a16-80e9-811112983f15 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.670202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.695192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.695788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.696599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.696705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.697083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.697332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.697436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.697878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] PATH_INFO: `/v3/users/89f68b5f957945ab9733376b4a928414` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.698386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e25e18c236c3471db6a7db6dce12b1fa) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:55.698767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.699669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] RBAC: Authorizing `identity:get_user(user_id=89f68b5f957945ab9733376b4a928414)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.700369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.701498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.704193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 101/202] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/users/89f68b5f957945ab9733376b4a928414 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:55.705574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.711854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 102/203] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/role_assignments?user.id=e25e18c236c3471db6a7db6dce12b1fa&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:55.712979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.830882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191a3f-59af-4d5d-a0b7-c1e2f71b4e10 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.873101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:55.874330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:55.874802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:55.875290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:55.876814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:55.878527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:55.881424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:55.888387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:55.917683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:55.963679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25b4b2-85e1-4c4a-9afa-da2198f2aba1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:55.989643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:56.017864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:56.018562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.019132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.019308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.045347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5bdf4-6ec4-4adb-b9e2-fe69af9ce799 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:56.088280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:56.095898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.096191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.096477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.097338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e25e18c236c3471db6a7db6dce12b1fa) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:56.097660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:56.099196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:56.117129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 103/205] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/role_assignments?user.id=e25e18c236c3471db6a7db6dce12b1fa&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:56.124367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:56.126272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:56.211501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:56.231892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 103/206] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:01:56.244292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:56.390495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010ae95-8e26-40fc-971a-ed32f44d1af5 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:56.407034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:56.435282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:56.436171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.436453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.436728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.442105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:56.443779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269efc68-c3d4-4913-9d7b-5be698beef1e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:56.445837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 104/207] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:56 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:56.453541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:56.608381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794179d-4f38-4d1d-90f5-a7855018aac3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:56.654671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:56.655374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.655665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.655939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.656785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688, user.id=89f68b5f957945ab9733376b4a928414) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:56.657098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:56.658579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1964-ca26-4942-9bd5-256715898d63 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:56.666349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 104/208] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:56 2022] GET /identity/v3/role_assignments?user.id=89f68b5f957945ab9733376b4a928414&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:56.948228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e345d-6e6b-4739-9873-21176ba67c15 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.948722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e345d-6e6b-4739-9873-21176ba67c15 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.948722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e345d-6e6b-4739-9873-21176ba67c15 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.950307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 105/209] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:56.955928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:56.956219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:56.956500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:56.977586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:56.978786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:57.006092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f82a-12a8-440b-8b89-1953c3b01017 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.006400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f82a-12a8-440b-8b89-1953c3b01017 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.006661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17f82a-12a8-440b-8b89-1953c3b01017 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.009480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 106/210] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:01:57.013620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.013918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.014203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.019861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:57.035445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:57.036802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:57.061511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 105/211] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:57.087626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.115296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 107/212] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:57.135206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.207174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bec6a-3cb0-492e-8981-2ae15f330a37 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.243884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.245033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.245486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.245956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.258008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283a1bd-1c5c-40d3-aa52-139cfea46fb8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.259259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:57.268042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:57.268942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:01:57.269728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:57.272957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 106/213] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:57.278701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.280049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:57.297160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.298348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.298727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.299012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.300267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:57.301730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:57.304820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 108/214] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:57.312029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.315558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:57.408190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bb97f-d65a-494e-a0e3-e967e78a94af None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.425983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a12852-8598-4a63-8c6f-1cf5eed51ce7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.458121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.459833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.460381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.460911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.462013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:57.462400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:57.463925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.464071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:57.464933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.465396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.465858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.466765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:57.468445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:57.472860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 107/215] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:01:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:57.479131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.497052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 109/216] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:01:57 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:01:57.615867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:57.629323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682e176-531e-4aed-b22d-4ca90d3c27fb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.669226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.670513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.671000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.671556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] PATH_INFO: `/v3/users/f433e4a96f79444e94270f7efa6159d1` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.688644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] RBAC: Authorizing `identity:get_user(user_id=f433e4a96f79444e94270f7efa6159d1)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:57.690483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:57.692753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 108/217] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:57 2022] GET /identity/v3/users/f433e4a96f79444e94270f7efa6159d1 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:57.698796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:57.741042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:57.816469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660866f-ee29-49e6-b6a4-76179e8a1eb8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:57.857921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:57.858604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:57.858888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:57.859195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:57.936119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:57.937620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:57.979178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:58.043630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 110/218] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:01:58.049215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.182418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c704719-32bb-49ac-b231-9626b8ccf783 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:58.217123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:58.218001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.218285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.218561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.219757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:58.221225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f49219-359f-47ff-93b8-44def0a6932e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:58.223325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 109/219] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:58.229322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.330644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae6d08-c11c-427b-8400-e037b1565410 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:58.363983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833ace8-fb0e-44c9-960b-ba25c500ae28 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.364435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:58.364578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833ace8-fb0e-44c9-960b-ba25c500ae28 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.364680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833ace8-fb0e-44c9-960b-ba25c500ae28 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.364773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.365111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.365304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.366226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 110/220] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:58.366358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=f433e4a96f79444e94270f7efa6159d1) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:58.366458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:58.367923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3bfbf-cc32-458f-9007-70f4b234ff81 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:58.371557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.371850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.372135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.378031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 111/221] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/role_assignments?user.id=f433e4a96f79444e94270f7efa6159d1&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=370601cf296e44ea95990467138eb5e5 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:58.404431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:58.405717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:58.421659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:58.502547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b473-30f7-4a3c-9a32-0d246b247f23 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.502849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b473-30f7-4a3c-9a32-0d246b247f23 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.503129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6b473-30f7-4a3c-9a32-0d246b247f23 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.504745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 112/222] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:01:58.507353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 111/223] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:01:58.510518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.510811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.511115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.525091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:58.531402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.544206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:01:58.545440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:01:58.648637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 113/224] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:01:58.668195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23077e72-f35b-4d1b-82ab-061259389863 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:58.670980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.719894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:58.720737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.721018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.721295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.745731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:58.755726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:58.756661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:01:58.757903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 112/225] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:01:58.767860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.788912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:58.859287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a90419-5852-4efc-8a56-f0e6097cb1fe None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:58.926497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fff14-45d3-44ff-909d-99928fb49ffe None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:58.928679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:58.943806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:58.944719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.944920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.945199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.965728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:58.966749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:58.967289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:58.967774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:58.968726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:58.969339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:58.970961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:58.984155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:01:58.984907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 113/226] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:58.986588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:01:58.990069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:01:58.990967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:01:58.991944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 114/227] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:01:59.004254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.079477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:59.094811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24756211-5985-414d-85b8-d68ac289daa1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.130158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.131553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.132041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.132534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.134045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.135777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.137657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 114/228] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:58 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:59.144006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.236227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0f4a-884f-4ea1-bad4-d2d93a7638d9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.279396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.280438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.280965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.281492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.282446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:01:59.282948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.284736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.297778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 115/229] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:59.301346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.330115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b14660-89a3-4c06-8529-34d610534dce None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.397207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.398537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.399087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.399904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.402833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.403441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:59.404749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.414798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 115/230] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:59 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:01:59.417513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.434953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:59.495723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b08283-22e4-461e-a4f2-a4c21addfcd1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.558395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.559697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.560210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.560682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.562735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.564432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.566944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 116/231] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:01:59 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:59.573170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.608700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1d42a-bcea-4f58-b3c3-93c4b0b708a6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.673483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.675218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.675720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e25e18c236c3471db6a7db6dce12b1fa/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.680579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, user_id=e25e18c236c3471db6a7db6dce12b1fa, role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.682303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.711320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 116/232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:59 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e25e18c236c3471db6a7db6dce12b1fa/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:01:59.766629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7489-22e7-40dd-b9a7-c6e21ebf6c34 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:01:59.810242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:59.825414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:01:59.826323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:01:59.826605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:01:59.826881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:01:59.828173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:01:59.829736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b55f9a-67f2-4893-b2d4-e939a5ded0c7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:01:59.831900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 117/233] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:01:59 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:01:59.837575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:01:59.929517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:01:59.968271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb44dc-67c3-47f6-8b3b-46427ceff362 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:00.003507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:00.004774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.005330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.005835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e25e18c236c3471db6a7db6dce12b1fa/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.008091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, user_id=e25e18c236c3471db6a7db6dce12b1fa, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:00.009845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:00.013545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6b59-c104-460a-9b59-313c69ea3fd2 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.014020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6b59-c104-460a-9b59-313c69ea3fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.014020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6b59-c104-460a-9b59-313c69ea3fd2 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.017786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 118/234] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:00.021816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.022531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.022944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.027779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 117/235] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:01:59 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e25e18c236c3471db6a7db6dce12b1fa/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:00.043292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:00.044573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:00.150831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 119/236] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:00.186004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:00.305807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f289c92-e1aa-4f5c-91a7-2c458011f618 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:00.333160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:00.344389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:00.345209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.345492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.345762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.357479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1583338f-2078-4cb4-94c4-0f25cbc3b939 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:00.366065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:00.366978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:00.368212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 118/237] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:00.373287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:00.522089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb063b89-d2f3-41a3-9094-50d8f36d44b4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:00.548304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:00.568933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82be8d-f8e4-4c8f-b4e7-fdcd8dacfe66 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.569550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82be8d-f8e4-4c8f-b4e7-fdcd8dacfe66 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.569902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:00.570297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82be8d-f8e4-4c8f-b4e7-fdcd8dacfe66 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.570603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.570890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.571191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.571966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:00.572276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:00.572946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 119/238] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:00.575167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:00.579110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.579634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.580128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.585651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 120/239] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:00.590588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:00.616939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:00.618406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:00.704712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 120/240] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:00.733829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:00.802661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c4402-3fce-4133-929f-f7481d8bb47e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:00.871126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:00.872940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.873540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.875274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] PATH_INFO: `/v3/users/e0574be9bd5145db89fd549f81667327` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.899003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] RBAC: Authorizing `identity:get_user(user_id=e0574be9bd5145db89fd549f81667327)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:00.901417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:00.906387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 121/241] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:00 2022] GET /identity/v3/users/e0574be9bd5145db89fd549f81667327 => generated 297 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:00.910172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:00.914845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d60d9-3b1f-46d0-a1a9-f9b1513a9074 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:00.948973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:00.961493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:00.984359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:00.985403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:00.985524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:00.985719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:00.991854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:00.993371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:00.999136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 121/242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:01.013054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.031696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39df1b-16ff-4a12-ad98-eff11cf50616 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.068438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.069438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.069945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.070413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.115474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bc041-04d6-4110-8406-120bd1603e37 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.141910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.143756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.149572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.150258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.150543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.150821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.151569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.153040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf009b51-8312-484e-90a3-2659c8f33b61 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.178166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 122/243] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:01 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:02:01.183229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83051f-9a6c-4de2-a0c6-50a08951af54 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.183229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83051f-9a6c-4de2-a0c6-50a08951af54 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.183229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83051f-9a6c-4de2-a0c6-50a08951af54 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.186002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 123/244] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:01.187604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.187712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.187974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.206111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:01.213808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:01.215187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:01.295509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:01.320452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 124/245] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:01.335591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 122/246] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:01.343407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.343593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.462933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdb9fd-1906-4542-9e74-d7f6418e90e3 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.511013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908251-3739-45df-9aea-afacf84e0461 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.516120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.517027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.517313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.517593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.518806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.520340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.522537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 125/247] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:01.528169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.554572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.556168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.556468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.556747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.567388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:01.573885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:01.582692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:01.583921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:01.584756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 123/248] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:01.591246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:01.596824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.674725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e58-134b-4f99-8d86-019928b21364 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.716760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.717373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.717631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.717918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.718768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e0574be9bd5145db89fd549f81667327) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:01.719109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.720626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.731153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 126/249] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/role_assignments?user.id=e0574be9bd5145db89fd549f81667327&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:01.747457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f947297-72df-405c-87f8-567110d91cb4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.790847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.791595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.791893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.792180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.792969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:01.793280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.794709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.807390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 124/250] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:01.812773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:01.941550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18d8e86-e9d3-4712-a5b7-9d7659352edb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:01.943877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:01.985532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:01.986364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:01.986649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:01.986929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] PATH_INFO: `/v3/users/f433e4a96f79444e94270f7efa6159d1` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:01.988247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] RBAC: Authorizing `identity:get_user(user_id=f433e4a96f79444e94270f7efa6159d1)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:01.989967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:01.991953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 127/251] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/users/f433e4a96f79444e94270f7efa6159d1 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:02.000484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:02.080623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:02.125032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32e83e-a8d9-41be-9405-70905f598767 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:02.174708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:02.175645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.175934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.176215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.178060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:02.179592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2774e6-9a47-487a-bd95-e3eb651f5731 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:02.181682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 125/252] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:01 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:02.187191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:02.314048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062fa0-ffc4-468e-ac61-61e854f5ac4c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:02.354639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:02.355586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.355874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.356207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/f433e4a96f79444e94270f7efa6159d1/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.358266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, user_id=f433e4a96f79444e94270f7efa6159d1, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:02.359790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96223a-0441-45e4-9169-d9a797fabf8a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:02.383822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 128/253] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:02 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/f433e4a96f79444e94270f7efa6159d1/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:02.422807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869082-2052-4366-9ec9-d4be76978699 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.423183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869082-2052-4366-9ec9-d4be76978699 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.423440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2869082-2052-4366-9ec9-d4be76978699 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.425087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 126/254] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:02.426525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54bd5-9d00-4a09-a2f5-4eea965d7215 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.427197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54bd5-9d00-4a09-a2f5-4eea965d7215 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.427197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc54bd5-9d00-4a09-a2f5-4eea965d7215 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.429177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 129/255] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:02.436823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.437022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.437129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.437388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.437858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.438343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.466621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:02.468292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:02.468592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:02.469511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:02.524551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:02.607839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 127/256] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:02.628623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 130/257] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:02.633466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:02.702158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:02.706369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:02.765957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e83af-0c15-482d-808b-ee48b0e3d580 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:02.847961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:02.849168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:02.849734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:02.850206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:02.884073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:02.899601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:02.900539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:02.901328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:02.904198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 128/258] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:02.910058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:02.957407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1dacf3-56e7-4b55-a411-95a2cb54e9a6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.030992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.031907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.032193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.032463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.075753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:03.084496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:03.085433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:03.086637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 131/259] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:03.087568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375b581-0628-48dc-bfc9-008b4ec4e498 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.091504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.115661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:03.144828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.146030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.146556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.147037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.148194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:03.148702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:03.150343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:03.154699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:03.160591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 129/260] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:03.167532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.288049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb95c44-53fc-42d5-8304-2f8fb57a1e7b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.326837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.328649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.329374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.329850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.331542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:03.333345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:03.339014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c5210-601f-482f-b6c5-9ac1ebc84816 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.339349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 130/261] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:03.341872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.382926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.383546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.383824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.384101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.384849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:03.385152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:03.386541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:03.400914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:03.407409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.446617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:03.535676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d0485f-e4df-464a-9363-5f34add9fc2d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.577378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.578063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.578350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.578628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.595475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:03.691466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:03.693111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:03.713051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2447ae2-a328-47f2-8569-ccda3485b0c9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:03.749261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:03.807490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 131/263] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:03.813444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.827829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:03.828746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:03.829034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:03.829316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] PATH_INFO: `/v3/users/e25e18c236c3471db6a7db6dce12b1fa` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:03.830560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] RBAC: Authorizing `identity:get_user(user_id=e25e18c236c3471db6a7db6dce12b1fa)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:03.832055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:03.834218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 133/264] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/users/e25e18c236c3471db6a7db6dce12b1fa => generated 283 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:03.839229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:03.979302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51243893-648d-4ce2-844e-484ff96b52f6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:04.034510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:04.035424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.035706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.036018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.037326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:04.038797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:04.040755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 132/265] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:04.046822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:04.054004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115c281-8d5b-4a70-9dd5-e36e4811e8bd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:04.079926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:04.128376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:04.129051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.129334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.129608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.169880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:04.201764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e7d3c-360c-4738-babc-1326ffd24621 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:04.262653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:04.263278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.263563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.263703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:04.263909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.264878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e25e18c236c3471db6a7db6dce12b1fa) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:04.265174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51866435-6b70-4e64-869b-35bd2c5e5cf3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:04.265278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:04.266730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:04.277732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 133/266] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:04 2022] GET /identity/v3/role_assignments?user.id=e25e18c236c3471db6a7db6dce12b1fa&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:04.402031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 134/267] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:04.407625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:04.516427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:04.542105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839a0f2-8348-4f9b-b95e-2bafd1ca1ebd None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:04.561354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734a8a-975e-421e-aeca-699c3818f85b None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.561686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734a8a-975e-421e-aeca-699c3818f85b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.561923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4734a8a-975e-421e-aeca-699c3818f85b None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.563603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 135/268] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:04.581645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.581645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.581645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.597221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:04.598122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.598408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.598762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.600250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:04.601738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:04.603858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 134/269] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:04 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:04.611642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:04.620223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:04.621462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:04.690699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:04.715134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 136/270] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:04.750512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486e890f-864a-4a3c-a758-a0cf6d7bd7f1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:04.750944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:04.813975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:04.813975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.814247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.814489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.815273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e25e18c236c3471db6a7db6dce12b1fa) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:04.815593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:04.817039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0562da-44fd-4cd2-8157-f4a4f8bbf267 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:04.823934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 135/271] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:04 2022] GET /identity/v3/role_assignments?user.id=e25e18c236c3471db6a7db6dce12b1fa&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:04.825265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87853d-3d95-4d01-838f-4a60c9832750 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.825552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87853d-3d95-4d01-838f-4a60c9832750 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.825831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87853d-3d95-4d01-838f-4a60c9832750 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.827135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 136/272] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:04.832801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2eaac-8a11-4811-9cd5-c08770f6d2a2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:04.833233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2eaac-8a11-4811-9cd5-c08770f6d2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:04.833605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2eaac-8a11-4811-9cd5-c08770f6d2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:04.867199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-23c2eaac-8a11-4811-9cd5-c08770f6d2a2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:04.873418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-23c2eaac-8a11-4811-9cd5-c08770f6d2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:04.893743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:04.929619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cd7e0-672a-4a2c-9eb1-2481266c2130 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.001037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.002401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.003835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.005152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.018560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 137/273] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:05.025006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93ec3-65da-4309-86bc-929ac2790ea1 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.025634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93ec3-65da-4309-86bc-929ac2790ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.026128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d93ec3-65da-4309-86bc-929ac2790ea1 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.027854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 138/274] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:02:05.035122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.035122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.035122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.040247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:05.048241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:05.049132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:05.049920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 137/275] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:05.051149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.057370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:05.058605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:05.101694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.135506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 139/276] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:05.136599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.181319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ec156-3943-4b24-9c54-2a0a5338c8d6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.221764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.227906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.228891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.229535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.229825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.243442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:05.247889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e9e9b-7de2-4c7e-b415-e59bb5d7e9fe None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:05.252598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:05.253481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:05.254333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 138/277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:05.256129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.284747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.285432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.285715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.286025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.286857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:05.287203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.288608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.302055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 140/278] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.302055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.366474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.369542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1f0ac-1c8b-443b-8c6f-f62588e87597 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.405492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.406304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.406594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.406876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.408107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.409591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.411418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 139/279] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.412274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.415220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2159df7-af3a-4c00-ba0c-4104e7e55885 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.458115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.459136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.459425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.459707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.460716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:05.461443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.463146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.466093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.474031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 141/280] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.474788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.514805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca26c4-8485-4b7c-bea4-cc4903db5c6f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.548267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.549157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.549442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.549719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] PATH_INFO: `/v3/users/32a0038d01594a94a3c5bb90b20fc6d4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.564748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] RBAC: Authorizing `identity:get_user(user_id=32a0038d01594a94a3c5bb90b20fc6d4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.566292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d55a1b-d4ae-4d10-bcf8-0914b4615c9f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.568160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 140/281] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/users/32a0038d01594a94a3c5bb90b20fc6d4 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.569401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540d103-d562-4bf0-bf64-a1cf41cdb1f9 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.569679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540d103-d562-4bf0-bf64-a1cf41cdb1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.569933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540d103-d562-4bf0-bf64-a1cf41cdb1f9 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.571315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 141/282] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:05.571859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540d103-d562-4bf0-bf64-a1cf41cdb1f9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.585628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ef212-b558-424d-bdb1-51f5cc89af76 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.598134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.620293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.620899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.621180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.621451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.622154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.623624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.646287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 142/283] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:05 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:05.647079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.685329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540d103-d562-4bf0-bf64-a1cf41cdb1f9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.718667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.728227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.729137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.729423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.729703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] PATH_INFO: `/v3/users/e0574be9bd5145db89fd549f81667327` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.730986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0574be9bd5145db89fd549f81667327)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.732549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e91ade-fc15-484b-ae51-22466353c4f7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.739225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 142/284] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/users/e0574be9bd5145db89fd549f81667327 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.741037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.741329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.741611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.758691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:05.759950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:05.774618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925ba0c-888e-4581-969b-513e534b3ba9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.816354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:05.817028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:05.817736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:05.818275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:05.819406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, user.id=32a0038d01594a94a3c5bb90b20fc6d4, scope.system=all) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:05.819509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:05.820905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:05.827833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 143/285] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/role_assignments?user.id=32a0038d01594a94a3c5bb90b20fc6d4&scope.system=all&role.id=370601cf296e44ea95990467138eb5e5 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:05.828657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.847761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 143/286] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:05.872767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:05.911070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:05.950085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eceefda-eeac-4de7-8edb-0da5afe03625 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:05.972947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:06.003569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.004635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.005110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.005400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.008120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:06.010181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:06.012393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 144/287] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:06.016849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febe87a-c13b-4c66-842d-a10894c5abb4 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.020357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:06.066451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.067638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.068003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.068382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.082424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:06.093198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:06.094142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:06.094936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:06.095580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 144/288] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:06.102319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:06.155847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548a290-a39a-4519-b07c-fb6005a5cbad None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.199130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.200088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.200373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.200653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e0574be9bd5145db89fd549f81667327/roles/e8be34dd9fd146969ddeabb56c1172b6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.203163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, user_id=e0574be9bd5145db89fd549f81667327, role_id=e8be34dd9fd146969ddeabb56c1172b6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:06.205096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:06.213066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:06.225093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 145/289] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:06 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/users/e0574be9bd5145db89fd549f81667327/roles/e8be34dd9fd146969ddeabb56c1172b6 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:06.236401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b561c87-6869-4006-83ec-801c712a6783 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.277819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.278579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.278870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.279224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.279998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:06.280310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:06.281865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:06.292395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 145/290] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:06.298488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:06.365266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:06.444323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542246e-3295-48bb-90bd-9526aed4c0b6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.481909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.482857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.483165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.483445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] PATH_INFO: `/v3/users/f433e4a96f79444e94270f7efa6159d1` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.484682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] RBAC: Authorizing `identity:get_user(user_id=f433e4a96f79444e94270f7efa6159d1)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:06.486147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:06.488142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 146/291] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:06 2022] GET /identity/v3/users/f433e4a96f79444e94270f7efa6159d1 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:06.493682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:06.601903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bbe23-79dd-4cc0-ad4b-77d87ac35ad0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.637871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.638538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.638825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.639125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:06.655707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:06.711140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:06.712649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:06.813661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 146/292] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:06.819195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:06.950963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5945152-9a2f-4447-83a5-c1067c6d4c90 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:06.968955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:06.998160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:06.999045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:06.999364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:06.999646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:07.000850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:07.002317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ece2ab-0ff8-4e70-b4c2-396a9b49c735 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:07.004070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 147/293] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:06 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:07.010273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:07.059734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:07.113183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96659ebb-d62a-433d-afed-e4794fc8f5b0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:07.150314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:07.151068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:07.151384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:07.151665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:07.152561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=f433e4a96f79444e94270f7efa6159d1) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:07.152877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:07.154355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcbef4-c448-42be-9f80-8d68ff417f5a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:07.161523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 147/294] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:07 2022] GET /identity/v3/role_assignments?user.id=f433e4a96f79444e94270f7efa6159d1&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=370601cf296e44ea95990467138eb5e5 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:08.167271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a263-cc2d-4f73-b8f1-44ccba27095c None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.168329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a263-cc2d-4f73-b8f1-44ccba27095c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.168840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a263-cc2d-4f73-b8f1-44ccba27095c None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.170695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 148/295] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:08.176276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.177903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.178442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.195446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:08.197104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:08.207314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7acd96-388b-41df-9807-bf806afc9ffd None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.207619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7acd96-388b-41df-9807-bf806afc9ffd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.207999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7acd96-388b-41df-9807-bf806afc9ffd None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.209547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 149/296] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:02:08.214292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.214611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.214862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.231755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:08.232932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:08.269260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 148/297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:08.291565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.306561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 150/298] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:08.330361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.335548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:08.409109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9143b745-2606-4a12-8440-9f8fd21b09e6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.437882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e7b37-e75f-43eb-a527-308b41564de8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.446474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.447321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.447812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.447916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.451669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:08.459613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4b665faf-b136-44cc-b4fd-280707b77c0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:08.467470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:08.468378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:08.469198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 149/299] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:08.475621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07430b-8dfb-4b5f-8501-60cee785fd0d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.475621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07430b-8dfb-4b5f-8501-60cee785fd0d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.475621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07430b-8dfb-4b5f-8501-60cee785fd0d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.475621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 150/300] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:08.476638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07430b-8dfb-4b5f-8501-60cee785fd0d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.484119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.484933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.485280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.485543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.486755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:08.488238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3c8cb-2fe9-4aae-beb3-6d05804ae18f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:08.489870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 151/301] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:08.491077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.491382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.491659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.508966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:08.510141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:08.587605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:08.588607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 152/302] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:08.589782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.591864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e07430b-8dfb-4b5f-8501-60cee785fd0d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.632320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.633010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.633297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.633575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.634332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:08.634644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:08.636075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:08.649712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:08.649712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.706358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839210a4-937a-4c5b-96be-c4cf762fba92 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.717324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:08.743899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.744885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.745417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.745699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] PATH_INFO: `/v3/groups` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.746546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:08.748573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:08.749826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cd6d6-daad-4a3f-931c-ece1daff6e48 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.771132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 153/304] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jun 30 04:02:08 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:08.771132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.786339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.787664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.787953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.788230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.800636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-817184af-9aec-4e25-9235-01efc7124dfc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:08.809089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:08.810159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:08.810969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:08.812131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386b75-1c34-4d3a-bf1e-a0f18e018eb5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.812361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386b75-1c34-4d3a-bf1e-a0f18e018eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.812550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23386b75-1c34-4d3a-bf1e-a0f18e018eb5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.813814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 153/306] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:08.816403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23386b75-1c34-4d3a-bf1e-a0f18e018eb5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:08.843889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:08.891768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c9b8d-72ac-42f9-af5e-0bdaf2c83899 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.936394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.937179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.937179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] PATH_INFO: `/v3/users/32a0038d01594a94a3c5bb90b20fc6d4` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.937365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23386b75-1c34-4d3a-bf1e-a0f18e018eb5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:08.938245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] RBAC: Authorizing `identity:get_user(user_id=32a0038d01594a94a3c5bb90b20fc6d4)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:08.939942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148be7b-f31d-4ece-9070-14ac57c800ad None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:08.942214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 154/307] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/users/32a0038d01594a94a3c5bb90b20fc6d4 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:08.943047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.943438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.943722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.961793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:08.963008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:08.978737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:08.979519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:08.979808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:08.980178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:08.980880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:08.981194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:08.982603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:08.991709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 154/308] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:08.992512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.000256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.041611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 155/309] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:09.042779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.086306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.103977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91b3df-8fe5-4ff0-bfba-8ade5c3f56db None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.146177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.147493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.147794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.148066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] PATH_INFO: `/v3/system/users/32a0038d01594a94a3c5bb90b20fc6d4/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.150056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=32a0038d01594a94a3c5bb90b20fc6d4, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.152050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:09.160514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab77425-c790-45d4-ab46-76272fc39fcd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.163610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 155/310] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jun 30 04:02:08 2022] PUT /identity/v3/system/users/32a0038d01594a94a3c5bb90b20fc6d4/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:09.164722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.222952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.225223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.225572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.226034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] PATH_INFO: `/v3/users/bcffb10d4551444eaad0ddfda292cfe2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.247037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] RBAC: Authorizing `identity:get_user(user_id=bcffb10d4551444eaad0ddfda292cfe2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.248582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:09.250938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 156/311] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/users/bcffb10d4551444eaad0ddfda292cfe2 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:09.258621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.288109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af7d2-3895-46e6-a16b-af0542811126 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.305258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.330019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.330857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.331170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.331453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.344669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:09.353643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:09.354542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:09.355740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 156/312] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:09.360827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.377485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.404475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fcf53-a9bf-404e-b26a-70593aa81fba None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.444384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.445217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.445566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.445916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.446883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, user.id=bcffb10d4551444eaad0ddfda292cfe2, scope.system=all) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:09.447371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.448896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:09.454936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 157/313] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/role_assignments?user.id=bcffb10d4551444eaad0ddfda292cfe2&scope.system=all&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:09.491856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bebb0-aae7-4108-9367-294e14dbb3f2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.530902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.531630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.531916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.532192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.532979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:09.533291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.534725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:09.544888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 157/314] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:09.550305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.663603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.673739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93630a91-91e2-409f-b92e-3dd5b0f8f194 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.712720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.713617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.713900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.714177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] PATH_INFO: `/v3/users/e0574be9bd5145db89fd549f81667327` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.715416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] RBAC: Authorizing `identity:get_user(user_id=e0574be9bd5145db89fd549f81667327)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.716869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:09.718831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 158/315] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/users/e0574be9bd5145db89fd549f81667327 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:09.724733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:09.794668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:09.829173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e8c2d-20bf-436e-af30-a53e5a91678a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:09.868099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:09.868819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:09.869106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:09.869429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:09.946329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:09.948606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:10.023839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:10.061529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 158/316] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:10.067150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:10.169680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207d31a-73c9-4000-90bb-1d7a8fededb7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:10.210949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:10.212310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:10.212793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:10.213286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:10.214820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:10.216539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826153f-e2c8-444b-80e5-336b804268e7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:10.218859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 159/317] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:10 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:10.224572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:10.339128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8859a-eabc-4aff-bde2-f8b2e62272bb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:10.379345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:10.380109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:10.380389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:10.380665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:10.381614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, scope.project.id=5852b172b7b94aa0a1ee688b7045df41, user.id=e0574be9bd5145db89fd549f81667327) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:10.381924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:10.383413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f2101-3a8a-4c36-bd02-82e39114545b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:10.390613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 159/318] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:10 2022] GET /identity/v3/role_assignments?user.id=e0574be9bd5145db89fd549f81667327&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:11.010191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191932-9b43-4e00-857d-7876f665b442 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.010740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191932-9b43-4e00-857d-7876f665b442 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.010740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191932-9b43-4e00-857d-7876f665b442 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.012440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 160/319] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:11.017481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.017740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.018206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.035521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:11.036797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:11.066183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:11.109730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 160/320] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:11.131985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191932-9b43-4e00-857d-7876f665b442 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.162383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:11.242189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191932-9b43-4e00-857d-7876f665b442 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:11.282499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:11.283270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.283555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.283829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.284998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:11.287030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d80e9c-4aaa-4389-bfb9-992ef3173c84 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:11.293228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 161/321] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:11.294462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.393827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5b41-5ced-4d97-b54e-63391f9ec021 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.394243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5b41-5ced-4d97-b54e-63391f9ec021 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.394522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5b41-5ced-4d97-b54e-63391f9ec021 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.395977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 162/322] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:02:11.400991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716ee63-05a7-4fd1-b3a1-383daf46401e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.401264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716ee63-05a7-4fd1-b3a1-383daf46401e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.401542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716ee63-05a7-4fd1-b3a1-383daf46401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.404343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5909ef1c-f91b-447f-b84f-0b9ea3097e10 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:11.421626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d716ee63-05a7-4fd1-b3a1-383daf46401e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:11.422795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d716ee63-05a7-4fd1-b3a1-383daf46401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:11.446549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:11.447201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.447487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.447769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] PATH_INFO: `/v3/groups` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.448646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:11.449873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:11.467557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 161/323] 10.222.0.13 () {60 vars in 1134 bytes} [Thu Jun 30 04:02:11 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:02:11.501372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 163/324] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:11.525389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.556533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:11.650991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd6916-3dca-4122-a584-c65558726aa7 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:11.652367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c70b-8d90-4e5e-8a7f-74273cd8a64d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.652668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c70b-8d90-4e5e-8a7f-74273cd8a64d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.653115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c70b-8d90-4e5e-8a7f-74273cd8a64d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.654493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 164/325] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:11.659804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.660095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.660371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.678555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:11.679795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:11.690355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:11.691330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.691621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.691898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.703227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:11.711421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:11.712446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:11.713406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:11.714065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:11.716109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 162/326] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:11.722635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.775203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 165/327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:11.791039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.835955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:11.851413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd32ee-f7ed-41fa-a0b0-6d4030790218 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:11.892314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:11.893006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.893296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.893569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.894336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:11.894642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:11.896486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:11.908495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 163/328] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:11.914204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf2504-97de-425c-8c2d-06999b9b5fb1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:11.916102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:11.960565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:11.961390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:11.961675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:11.961952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:11.972686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:11.980742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:11.981705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:11.987678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 166/329] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:11.992863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:12.013770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:12.036454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828eacd6-bf6a-4281-98e0-1d7e3ca222ca None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:12.075359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:12.076308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:12.076757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:12.077179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] PATH_INFO: `/v3/users/32a0038d01594a94a3c5bb90b20fc6d4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:12.078589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] RBAC: Authorizing `identity:get_user(user_id=32a0038d01594a94a3c5bb90b20fc6d4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:12.080241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:12.082126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 164/330] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/users/32a0038d01594a94a3c5bb90b20fc6d4 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:12.088772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:12.106649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:12.123535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178d155-79c4-4af6-a043-d434b08bcfda None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:12.174757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:12.175895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:12.176465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:12.177542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:12.178601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:12.179135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:12.182766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:12.193235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 167/331] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:12.199869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:12.206557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a7c9-55f2-4645-b1a3-936d1760b968 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:12.243037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:12.244156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:12.244638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:12.245107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:12.246140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] RBAC: Adding query filter params (role.id=370601cf296e44ea95990467138eb5e5, user.id=32a0038d01594a94a3c5bb90b20fc6d4, scope.system=all) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:12.246646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:12.248332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:12.267227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 165/332] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 30 04:02:12 2022] GET /identity/v3/role_assignments?user.id=32a0038d01594a94a3c5bb90b20fc6d4&scope.system=all&role.id=370601cf296e44ea95990467138eb5e5 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:12.304636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:12.337775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e36cb2-7ca9-41e9-8355-b681e0d90fb8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:12.380545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:12.381482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:12.381816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:12.382101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] PATH_INFO: `/v3/users/bcffb10d4551444eaad0ddfda292cfe2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:12.383571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] RBAC: Authorizing `identity:get_user(user_id=bcffb10d4551444eaad0ddfda292cfe2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:12.385069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e1924-8782-49bf-b8e8-55872519a90d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:12.387000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 168/333] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:12 2022] GET /identity/v3/users/bcffb10d4551444eaad0ddfda292cfe2 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:12.393067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:12.478054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:12.510015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d16a2-2b51-447f-a21b-9ec65ce0e11b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:12.545138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:12.546024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:12.546312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:12.546595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] PATH_INFO: `/v3/system/users/bcffb10d4551444eaad0ddfda292cfe2/roles/e8be34dd9fd146969ddeabb56c1172b6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:12.548163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bcffb10d4551444eaad0ddfda292cfe2, role_id=e8be34dd9fd146969ddeabb56c1172b6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:12.549570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e4494-88a9-4f44-bb7e-e9abd1f3f3e1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:12.565518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 166/334] 10.222.0.13 () {58 vars in 1334 bytes} [Thu Jun 30 04:02:12 2022] PUT /identity/v3/system/users/bcffb10d4551444eaad0ddfda292cfe2/roles/e8be34dd9fd146969ddeabb56c1172b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:13.852807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd40c-20c4-47a8-830d-14eed86c2789 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:13.853800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd40c-20c4-47a8-830d-14eed86c2789 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:13.854285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dd40c-20c4-47a8-830d-14eed86c2789 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:13.856129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 169/335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:13.861802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:13.862627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:13.863212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:13.884801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:13.886005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:13.982570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 167/336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:14.010443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd40c-20c4-47a8-830d-14eed86c2789 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:14.158214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dd40c-20c4-47a8-830d-14eed86c2789 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:14.196297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:14.197124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.197410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.197685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.214524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e9c1ac59-bcc7-46b0-acbf-12272afbf678 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:14.222536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:14.223555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:14.224352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 170/337] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:14.233597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:14.243998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a77d3-740d-4a96-8d80-bd002ba4bfab None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.244293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a77d3-740d-4a96-8d80-bd002ba4bfab None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.244594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a77d3-740d-4a96-8d80-bd002ba4bfab None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.246819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 171/338] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:14.252337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.254899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.254899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.270788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:14.282275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:14.287998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:14.371464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:14.452752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 172/339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:14.455676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f565deb-3b31-47bf-8bb9-13f5289f351d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:14.509268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:14.509954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.510230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.510501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.511296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] RBAC: Adding query filter params (name=member) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:14.511607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:14.513030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:14.518291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:14.541643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 168/340] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:14.557304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:14.745289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaa4a1-79b2-44f0-a40e-72e98b744344 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:14.745501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342d814-9c30-4404-b826-9f54a1204486 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:14.779551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:14.801933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:14.803412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.803702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.803984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.809869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:14.810701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.810981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.811280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.816204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:14.833776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-267c5ec1-1c18-4eb1-961c-cb955dd6f5e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:14.842339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:14.843287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:14.844076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:14.846551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 173/341] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:14.847958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6895a-ebaa-4337-b519-9914ecca382a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.848408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6895a-ebaa-4337-b519-9914ecca382a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.849403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6895a-ebaa-4337-b519-9914ecca382a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.851167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 174/342] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:14.856289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:14.857269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:14.857733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:14.892125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:14.893343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:14.903828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:14.908745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.010205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 175/343] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:15.011394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.037052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 169/344] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:02:15.044886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.091001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.150180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a376db-c5bd-4878-8d6c-8728b6bc304d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.175498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.187477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.188388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.188900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.189347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.190256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:15.190706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:15.192650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.201711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 176/345] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:15.202756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.220371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c4d90-003e-4903-adf4-5f242f626f69 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.280795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.281911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.282338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.282770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.284182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:15.285840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.291976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 170/346] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:15.292996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.311714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00cdd-6929-40dd-a21e-f6d65650fbe0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.347068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.348148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.348722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.349172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.360753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:15.369040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:15.370170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:15.371223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:15.373259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 177/347] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:15.374177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.432921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.442829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.449523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7290a-ede5-4445-aca1-6e1df15bcb12 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.491386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c38ed1-f28b-46e1-95a4-83f395609c19 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.521683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.522613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.523037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.523560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.531483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.532690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.533123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.533546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.543509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:15.545399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.547509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 178/348] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:15.548882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.641168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:15.642917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.699662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a22983-1dc2-48bc-9687-34f4251f5dc2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:15.711484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.750867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:15.751590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:15.751880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:15.752164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:15.752929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:15.753245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:15.754733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:15.773831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 179/349] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:15.774740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.827183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:15.833811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 171/350] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:15.839840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:15.979480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745feda0-3acb-478c-86cc-14db31e8802a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.019342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47f5ef-3e23-4b85-bdb1-7fff87cad8f3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.020384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.021175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.021525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.021808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.022800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:16.023165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.024762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.033317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 180/351] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=370601cf296e44ea95990467138eb5e5 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:16.034351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.075464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.084617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.085609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.086072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.086522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.175201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.177736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d483be0-5079-44e4-be88-f35382df176b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.186660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.188623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325545-7817-4fc9-8fd6-521c3afc94db None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.222436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.223917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.224416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.224945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.226497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.228188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0482191-669a-4ea1-bb8b-94858faf4c89 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.230310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 181/352] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:16.231977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c584b-84af-42f0-b292-0af127f7e69b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.232472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c584b-84af-42f0-b292-0af127f7e69b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.232892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c584b-84af-42f0-b292-0af127f7e69b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.234637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 182/353] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:16.240302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496facce-3431-437a-8766-95869df2cc41 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.240940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496facce-3431-437a-8766-95869df2cc41 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.241366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496facce-3431-437a-8766-95869df2cc41 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.243033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 183/354] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:16.244532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496facce-3431-437a-8766-95869df2cc41 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.347236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 172/355] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:16.348984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75adfe-21e7-4240-aa35-00e4d6cd88a9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.349325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75adfe-21e7-4240-aa35-00e4d6cd88a9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.349602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75adfe-21e7-4240-aa35-00e4d6cd88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.369401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6e75adfe-21e7-4240-aa35-00e4d6cd88a9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:16.370663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6e75adfe-21e7-4240-aa35-00e4d6cd88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:16.383202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496facce-3431-437a-8766-95869df2cc41 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.386792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.433103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.434281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.434574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.434857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.436088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.438152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195bbaa-645b-48eb-8514-4b9fecd5f64d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.440594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 184/356] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:16.441887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.442379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.443046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.453198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 173/357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:16.454521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9ac4-2285-4ae2-b33e-3b627afcc73c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.454867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9ac4-2285-4ae2-b33e-3b627afcc73c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.455182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9ac4-2285-4ae2-b33e-3b627afcc73c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.456543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 174/358] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:16.457295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9ac4-2285-4ae2-b33e-3b627afcc73c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.467546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:16.469296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:16.490054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.560702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 185/359] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:16.562255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.591821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9ac4-2285-4ae2-b33e-3b627afcc73c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.633676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.634596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.634885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.635194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.637384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.639015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b735f-4d0c-48e4-83aa-7a9460eae12f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.640859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 175/360] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:16.642045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.643430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.643742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.663538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:16.664909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:16.716018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec16994-e30c-4991-adba-bec67a17fa31 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.717896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.747592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 176/361] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:16.748778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.771486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.773479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.773777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.774161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.775355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:16.775659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:16.775983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:16.777709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:16.784657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 186/362] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:16.786819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.859420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8432f-7dcc-4b39-b4f3-51c1b276ef36 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.900177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.900622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.900912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.901196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.914637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f77b18d-a16d-4429-bc3e-3da850a13a0f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:16.915722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:16.924221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:16.925213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:16.926005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 177/363] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:16.926774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:16.959375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:16.960721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:16.961258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:16.961731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:16.974309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:16.983884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:16.985259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 04:02:16.986087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:16.988943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 187/364] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:16.990038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.025079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.047557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.064102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4fd35b-6203-41b1-8ddd-d740f07cc645 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.114169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.115703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.116293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.117105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.118802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.121155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.123473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7781a-a7a7-480e-9290-201fcb97834f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.124042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 178/365] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.125179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.166486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.167337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.167690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.168031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.181526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:17.191489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:17.192799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:17.193600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 188/366] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:17.194249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.257075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9887479-c382-41aa-b669-3faffa0d597f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.294441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.295452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.295918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.296393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.297387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:17.297867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.299540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.301768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc54a8-d69f-4218-8789-77982a5dd6b7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.310736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 179/367] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.311912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.320727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.334445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.341599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.342393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.342740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.343081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.344751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:17.345160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.346662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.356780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 189/368] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.362314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.435988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff04799-192e-4231-8c01-f7bc6c2deb0f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.467503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270625bf-d7bd-486f-8b37-3cc581780620 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.471431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.472357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.472840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.473658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.474678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=5852b172b7b94aa0a1ee688b7045df41) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:17.475189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.476830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.482864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 180/369] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=370601cf296e44ea95990467138eb5e5 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.484023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.508274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.508979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.509262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.509538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.510590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:17.510774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.512217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.522488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 190/370] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.523505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.537054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.579602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.609795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c07398-2077-4621-b03c-211cb8380810 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.643373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badaadd-4c4a-4863-af8f-f59914b2e689 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.659443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.660214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.660854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.661199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.686748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.687995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.688458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.688903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] PATH_INFO: `/v3/users/bcffb10d4551444eaad0ddfda292cfe2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.690566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] RBAC: Authorizing `identity:get_user(user_id=bcffb10d4551444eaad0ddfda292cfe2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.692537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.695133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 191/371] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/users/bcffb10d4551444eaad0ddfda292cfe2 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:17.696229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.745284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.746931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:17.800074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.813004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90e6df-1f41-4114-9f74-fbe365b7c7fc None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.833171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:17.858620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:17.859679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:17.860176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:17.860636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:17.863308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 181/372] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:17.864439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:17.996244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:17.997803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717f85b-fc50-4bb2-9901-ee0ef76656f2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:17.997934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.037991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.039275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.039589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.039873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.040728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] RBAC: Adding query filter params (role.id=e8be34dd9fd146969ddeabb56c1172b6, user.id=bcffb10d4551444eaad0ddfda292cfe2, scope.system=all) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:18.041035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.042499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.057347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 182/373] 10.222.0.13 () {56 vars in 1308 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/role_assignments?user.id=bcffb10d4551444eaad0ddfda292cfe2&scope.system=all&role.id=e8be34dd9fd146969ddeabb56c1172b6 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:18.058208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.114291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:18.191581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:18.214002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 192/374] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:18.216488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68326054-6aa5-45c3-b5be-35c69bc7b0db None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.220277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.284082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.284893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.285177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.285460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.292040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.293555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.295758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 183/375] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:18.301574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.358663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05815f9-1607-4118-a804-0a43a7fcd247 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.395968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.396986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.397280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.397556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] PATH_INFO: `/v3/projects/447338b8480048d8be1a3dac45927add` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.398908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] RBAC: Authorizing `identity:get_project(project_id=447338b8480048d8be1a3dac45927add)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.400628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.402831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 193/376] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/projects/447338b8480048d8be1a3dac45927add => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:18.407957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.464371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:18.487167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1a4b-71c4-4a2e-a238-682306dfb5d2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.490147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:18.527404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5bb1a-90b3-4e29-942b-3000d1a349c0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.559294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.560110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.560400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.560675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.561899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.563640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.566729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.566867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 184/377] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:18.566997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.566997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.566997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] PATH_INFO: `/v3/groups/a67bcab61c334e67bb82d63b57a5adb9` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.574851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.576108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] RBAC: Authorizing `identity:get_group(group_id=a67bcab61c334e67bb82d63b57a5adb9)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.581919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.589477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 194/378] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/groups/a67bcab61c334e67bb82d63b57a5adb9 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:18.597165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:18.689358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac1e5f-c9f8-4f9d-abe3-975c65cab445 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.725197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.726242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.726745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.727253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.728287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=5852b172b7b94aa0a1ee688b7045df41) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:18.728790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.730427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9b6f-36a3-435f-8a27-540d9d8878d0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.737018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 185/379] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:18.757493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dab3ec-f37b-4f8e-9b39-a181925b6101 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:18.796719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:18.807932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:18.808629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.808913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.809190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.810097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] RBAC: Adding query filter params (group.id=a67bcab61c334e67bb82d63b57a5adb9, role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=447338b8480048d8be1a3dac45927add) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:18.810356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:18.811860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758e379-3ae1-47f0-a0f3-cbe17e0fbf83 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:18.819222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 195/380] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:18 2022] GET /identity/v3/role_assignments?group.id=a67bcab61c334e67bb82d63b57a5adb9&scope.project.id=447338b8480048d8be1a3dac45927add&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:18.941911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b715d6-d005-4ed2-a869-b06208997528 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.942694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b715d6-d005-4ed2-a869-b06208997528 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.943226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b715d6-d005-4ed2-a869-b06208997528 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.945226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 186/381] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:18.950924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:18.951514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:18.951974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:18.975266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:18.976911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:19.068633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 196/382] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:19.110102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b715d6-d005-4ed2-a869-b06208997528 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.159020 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:19.232362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b715d6-d005-4ed2-a869-b06208997528 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:19.275377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:19.276192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.276475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.276745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.294007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:19.305733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:19.306681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:19.307573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 187/383] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:19.313124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.426454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83925ef6-0679-42c6-a74c-a5382006518f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:19.429443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:19.466521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:19.467246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.467534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.467814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.468574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:19.468880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:19.470273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141618f4-9562-42fa-8b9e-32160d4e0737 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:19.482266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 197/384] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:19.487937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.534445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffab32-4dfb-46c8-b436-f1e05d3c6435 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.534796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffab32-4dfb-46c8-b436-f1e05d3c6435 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.535014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffab32-4dfb-46c8-b436-f1e05d3c6435 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.537305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 198/385] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:19.542375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.542668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.542948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.580159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:19.581607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:19.624407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:19.631739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ffd8-fe0d-431a-8967-9098866b114e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:19.704383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:19.705178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.705458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.705732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.707816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:19.709393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:19.711039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 188/386] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:19 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:19.718995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.720752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 199/387] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:19.777315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.865704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f74d-f391-4d65-8969-570161799520 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:19.904439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:19.905330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.905623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.905907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:19.907153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:19.908634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:19.911035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 189/388] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:19 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:19.917220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:19.921949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:19.933066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8544f4-5450-43a8-98a9-048fa9cdc25b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:19.966951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:19.995818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:19.996665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:19.996957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:19.997240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.024904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:20.031915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac2428-3f9c-47f7-93c0-26b6d8c3b890 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:20.034774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:20.039804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:20.040607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:20.041641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 200/389] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:20.047497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:20.095904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.096818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.097104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.097381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.102314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, group_id=88ade5a440b14a90a70f68efdc5fb5d2, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:20.104641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:20.133554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 190/390] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 30 04:02:19 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:02:20.206476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a7752-12a4-431d-ae5a-43adddbfe687 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.248315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.248948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.249235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.249518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.250372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:20.250685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:20.252418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:20.263085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 201/391] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:20.269003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:20.374839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:20.384465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52beb472-5ac5-4681-8db9-c21b53c29706 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.423577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.424467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.424756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.425037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.426257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:20.428341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036927a7-c66b-483b-8e2a-48797dc01287 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:20.430382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 191/392] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:20 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:20.435698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:20.463507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:20.565948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209813c-6221-42b2-b467-8b08075c6e28 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.594625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d781-60a3-4ed9-8c79-e96b28f095dd None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.595250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d781-60a3-4ed9-8c79-e96b28f095dd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.595746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937d781-60a3-4ed9-8c79-e96b28f095dd None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.597533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 192/393] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:20.604381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.605056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.605528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.608924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.609824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.610115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.610397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.611627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:20.613095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:20.615600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 202/394] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:20 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:20.621359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:20.627487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:20.628919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:20.706895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 193/395] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:20.735353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:20.750317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f9c3b-00d8-4c27-ad81-5d4efe560f6e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.752221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:20.823558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.824712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.825145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.825562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.829269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] RBAC: Authorizing `identity:create_grant(project_id=1bdbc3b6409f4ac0ab6a748ee8998688, group_id=88ade5a440b14a90a70f68efdc5fb5d2, role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:20.831632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:20.858180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 203/396] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 30 04:02:20 2022] PUT /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:02:20.887152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:20.897175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f0d20-2677-45e2-ae73-a2a86541e026 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:20.944060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:20.945225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:20.945716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:20.946145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:20.961999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:20.970636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:20.971922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:20.972911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:20.975227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 194/397] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:20.981488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:21.117429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2987541-6790-4b8b-903b-ede9d815794a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:21.157615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:21.158313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.158607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.158890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.159697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:21.160078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:21.161544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:21.170938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 204/398] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:21.176585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:21.290633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e496c2e-3243-464e-9cce-2e7a479d2a88 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:21.331784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:21.332702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.332987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.333269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.334590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:21.336090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:21.338050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 195/399] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:21 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:21.343571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:21.356107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:21.453513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6748a8-edc4-4262-80c6-5f7c7fa3877a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:21.489039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:21.489938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.490225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.490503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.491747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:21.493210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550087-24f2-4eba-9381-10f2670a725f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:21.495111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 205/400] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:21 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:21.500600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:21.521116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:21.579320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dee7-5921-40b7-85a3-57eed60c9e29 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.579625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dee7-5921-40b7-85a3-57eed60c9e29 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.579889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9dee7-5921-40b7-85a3-57eed60c9e29 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.581317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 206/401] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:21.592791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e10-f02c-408a-87f4-7bf7b6c01cfb None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.593087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e10-f02c-408a-87f4-7bf7b6c01cfb None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.593372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a6e10-f02c-408a-87f4-7bf7b6c01cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.622362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-471a6e10-f02c-408a-87f4-7bf7b6c01cfb None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:21.623612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-471a6e10-f02c-408a-87f4-7bf7b6c01cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:21.641162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba9248-95e2-4708-9672-42b42bb8c446 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:21.703618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:21.704453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.704735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.705013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.708027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, group_id=88ade5a440b14a90a70f68efdc5fb5d2, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:21.709662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bbd0a-47b6-45b7-9eeb-b9a20146f000 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:21.742447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 196/402] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 30 04:02:21 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:21.745832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 207/403] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:21.770050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa3ad-73ff-4d31-aec4-f20dae92b776 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.770392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa3ad-73ff-4d31-aec4-f20dae92b776 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.770599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2fa3ad-73ff-4d31-aec4-f20dae92b776 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.772593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 197/404] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:21.777861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.778152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.778428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.784101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fa3ad-73ff-4d31-aec4-f20dae92b776 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:21.797913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:21.812173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:21.813385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:21.895500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:21.921158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2fa3ad-73ff-4d31-aec4-f20dae92b776 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:21.961073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:21.961961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:21.962265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:21.962534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:21.977075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:21.985811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:21.986786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:21.987623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:21.989191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 198/405] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:21.993440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.022064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 208/406] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:22.046111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.117104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c60b70-cb32-4eb3-bbbd-a6b190d50776 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.174478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.175191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.175474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.175752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.176532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:22.177188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.178662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.184084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:22.189125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 199/407] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:22.199847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1df15d-00a0-4928-b8cb-55ed84ada768 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.200154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.247245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.247995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.248280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.248560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.255331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:22.270442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:22.284102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:22.285006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:22.285859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:22.287652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 209/408] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:22.297096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.381025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1965484-822b-4b4d-82ac-f22907c67573 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.430200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.431112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.431397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.431675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] PATH_INFO: `/v3/groups/a67bcab61c334e67bb82d63b57a5adb9` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.432920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] RBAC: Authorizing `identity:get_group(group_id=a67bcab61c334e67bb82d63b57a5adb9)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.434379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.436314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 200/409] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/groups/a67bcab61c334e67bb82d63b57a5adb9 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:22.445540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.494050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931b95a-a42b-4dc5-8156-25c25201ebce None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.540013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.540629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.540911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.541186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.541956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:22.542265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.545216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.566342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 210/410] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:22.572393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.591121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:22.649039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7ec28-6e7a-45e8-978f-d660070a8f6e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.688325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:22.717378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.718315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.718604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.718883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] PATH_INFO: `/v3/projects/447338b8480048d8be1a3dac45927add` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.722046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] RBAC: Authorizing `identity:get_project(project_id=447338b8480048d8be1a3dac45927add)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.723653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.727215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 201/411] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/projects/447338b8480048d8be1a3dac45927add => generated 344 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:22.730863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.789095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1767797-c592-434e-a44d-eae21fdf8d71 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.852885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.853690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.853974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.854255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.859784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.862236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.864000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 211/412] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:22.866875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a0b6f-fe3d-4a60-b2b2-1d9bf0fd8354 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:22.870736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:22.932967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:22.933878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:22.934168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:22.934447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] PATH_INFO: `/v3/projects/447338b8480048d8be1a3dac45927add/groups/a67bcab61c334e67bb82d63b57a5adb9/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:22.936467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] RBAC: Authorizing `identity:create_grant(project_id=447338b8480048d8be1a3dac45927add, group_id=a67bcab61c334e67bb82d63b57a5adb9, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:22.937994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391438d2-c661-4517-bfb4-3ae06b690f05 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:22.943407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:22.962345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 202/413] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 30 04:02:22 2022] PUT /identity/v3/projects/447338b8480048d8be1a3dac45927add/groups/a67bcab61c334e67bb82d63b57a5adb9/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:02:23.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582c869-75f6-40bc-913d-0f92b34fd563 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:23.021650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206f82-8ebc-41cb-b722-407da181035d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.021970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206f82-8ebc-41cb-b722-407da181035d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.022285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206f82-8ebc-41cb-b722-407da181035d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.023999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 203/414] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:02:23.033028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.033241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.033546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.047601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:23.069425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:23.071869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.073197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.073738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.077222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:23.081194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:23.082471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:23.086337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:23.086900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 212/415] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:22 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:23.090430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:23.183811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 204/416] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:23.208480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:23.238199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d0aed-06aa-4765-8b55-3c2dd793484c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:23.283299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:23.285189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.286210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.286710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.289695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] RBAC: Authorizing `identity:create_grant(project_id=5852b172b7b94aa0a1ee688b7045df41, group_id=88ade5a440b14a90a70f68efdc5fb5d2, role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:23.291625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:23.323053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 213/417] 10.222.0.13 () {58 vars in 1442 bytes} [Thu Jun 30 04:02:23 2022] PUT /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41/groups/88ade5a440b14a90a70f68efdc5fb5d2/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:02:23.348519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df45b8f8-4beb-4983-a87b-815329a3661a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:23.378151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:23.387447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:23.388279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.388613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.388896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.400781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-121b19d2-5e4f-4be8-bdf1-1d1fc0d5ba49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:23.408843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:23.409748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:23.411208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 205/418] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:23.416630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:23.513782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470a03-c759-481b-a512-378d99410c23 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.514076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470a03-c759-481b-a512-378d99410c23 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.514512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef470a03-c759-481b-a512-378d99410c23 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.516577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 206/419] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:23.517782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:23.523541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.524751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.525133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.556132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:23.556798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:23.587638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba8ad-7a62-4195-8ead-e3c8327f1f55 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:23.625799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:23.626508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.626793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.627060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.627856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:23.628157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:23.629683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:23.638638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 214/420] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:23.646331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:23.655188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 207/421] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:23.698712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:23.752467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00dcdc-4765-4719-90ff-efd780542943 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:23.775763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:23.791507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:23.792177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:23.792465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:23.792745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:23.866818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:23.868360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:23.976376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 215/422] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:23.984062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.003580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.020525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080b705-91b5-4da4-aaf7-bc639a1d17a9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.054345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.089447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16010be-1eba-4b0e-8fd9-cf847f004d5d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.090756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.091727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.092022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.092306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.109675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:24.125862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:24.126454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:24.127367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:24.128247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 208/423] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:24.130030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.130588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.131047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.132554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.133477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.134418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.136792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 216/424] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:23 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:24.148944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.238040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.256771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ed9ee-c1d3-425f-9cb3-f76d9e3eedda None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.294165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.295488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.295963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.296525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.297926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.299271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33010b71-c1c7-4f0d-953f-69556959efa1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.300170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f97be60-583a-40d7-8b11-78532af81b97 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.303001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 217/425] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:24.304477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b47-6d83-4230-b785-3e162f602991 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.304908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b47-6d83-4230-b785-3e162f602991 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.305314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b584b47-6d83-4230-b785-3e162f602991 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.306922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 218/426] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:24.308258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b584b47-6d83-4230-b785-3e162f602991 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.344241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.345134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.345775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.346154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.348040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:24.348528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.350039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4627f6-c779-4dbf-8588-5783ddd741fb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.361052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 209/427] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:24.363970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.364404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.364881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.391426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:24.392644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:24.428520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b584b47-6d83-4230-b785-3e162f602991 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.433416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.472696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.474099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.474557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.475050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.475557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 210/428] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:24.476268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:24.476673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.476957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.478577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.487536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 219/429] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=370601cf296e44ea95990467138eb5e5 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:24.502851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.523531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.590674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b17d1-5301-44fb-bfcb-fa0638388a0b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.615949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e88b-b602-40ad-914e-3057952bbd67 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.639242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.639914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.640195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.640470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.653436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.654631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.655153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.655656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.667275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:24.675483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:24.676522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 04:02:24.677684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:24.680482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 220/430] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:24.686406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.708746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.769609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.770705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:24.780481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.793221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e66b4a-d3f8-4fd9-afb9-d17be1865021 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:24.832333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:24.833543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:24.834025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:24.834491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:24.835477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] RBAC: Adding query filter params (name=member) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:24.835982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:24.837578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:24.847287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 221/431] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:24.856327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:24.921702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 211/432] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:24.930080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.004611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159b0c-3dea-4788-bad1-86bdb7357954 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.026747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.050586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.054329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.054857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.055400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.066505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.069066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80baf8-d51a-46b3-b47b-3c4dbb7c72ea None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.112238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.113486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.114018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.114503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] PATH_INFO: `/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.115943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] RBAC: Authorizing `identity:get_project(project_id=1bdbc3b6409f4ac0ab6a748ee8998688)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.117593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.119811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 212/433] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/projects/1bdbc3b6409f4ac0ab6a748ee8998688 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.127235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.145453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.147637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.242283 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bf5ea-6dbc-42e8-8ed2-b02e2ce40156 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.270651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 222/434] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:02:25.276532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.287574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.288922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.289655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.290127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.293702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.295437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.297908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 213/435] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.307299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.325307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.334650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.408102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7676-90f6-47f1-b878-55bb1bf29cde None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.443485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ce4ee-cf37-4d99-9894-2f4746e0f501 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.450199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.451373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.451662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.451942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.453141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.454590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31c9296-e6a2-43f0-8ac9-b932a90f70e9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.460741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 223/436] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.460741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92fcb9-f46f-4786-8c13-484cb7e615f1 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.460741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92fcb9-f46f-4786-8c13-484cb7e615f1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.460741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92fcb9-f46f-4786-8c13-484cb7e615f1 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.460741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 224/437] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:25.464543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92fcb9-f46f-4786-8c13-484cb7e615f1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.484019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.484682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.484972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.485252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.486090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:25.486411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.487975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d5df-e340-4d38-8a73-01dc454ebf3c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.503336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 214/438] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=1bdbc3b6409f4ac0ab6a748ee8998688&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.505081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.505382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.505652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.529198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:25.530733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:25.593040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92fcb9-f46f-4786-8c13-484cb7e615f1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.603752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.635600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.639648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.640898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.641380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.641483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 215/439] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:25.641938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.643933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.645744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.648713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 225/440] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.655431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.670622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.773126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7351d-90f9-41b9-b6c4-2285a3de952d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.789138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf910ff-9511-45de-be78-97b4b4b1c19f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:25.811909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.812670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.812999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.813281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.814149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=370601cf296e44ea95990467138eb5e5, scope.project.id=5852b172b7b94aa0a1ee688b7045df41) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:25.814465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:25.817004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:25.824168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 216/441] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=370601cf296e44ea95990467138eb5e5 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:25.831298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:25.832606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.833088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.833639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.846950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b86d8292-bbd9-4f82-b95c-3c9ff7989149 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:25.856174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:25.857163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:25.857960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:25.861009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 226/442] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:25.867399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:25.947782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:25.954593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209402b1-40ae-4d44-a11f-65e210b37d89 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.954938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209402b1-40ae-4d44-a11f-65e210b37d89 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.955230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209402b1-40ae-4d44-a11f-65e210b37d89 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.958300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 227/443] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:25.962929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:25.963262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:25.963546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:25.987841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:25.989103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:25.995815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5c1b5-873c-4134-b958-9fd22e6b22c2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.018029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.035026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.035026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.035654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.035762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.036565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:26.036947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.038589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.048561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 217/444] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:26.054473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.071160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 228/445] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:26.093077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.172824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bb3f9-e5be-437b-bec2-1301d336390b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.204467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.215817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.216096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865afe8-5b29-4f5c-98c8-6a4ad30e522d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.216479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.216766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.217047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.249211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.250035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.250325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.250607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.261675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:26.269835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:26.270783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:26.271622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 229/446] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:26.276823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.284399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.285906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.301277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.375642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143563ae-69be-4d4c-b366-3e8635df512d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.383934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 218/447] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:02:26.389561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.409639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.410318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.410685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.411033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.411860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:26.412178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.412319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.413675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.421817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 230/448] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:26.427853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.488919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe1e1b-3473-443b-af93-e87a072c26cb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.511668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.523785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.524660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.524970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.525244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] PATH_INFO: `/v3/projects/447338b8480048d8be1a3dac45927add` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.526462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] RBAC: Authorizing `identity:get_project(project_id=447338b8480048d8be1a3dac45927add)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.526817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02f688-28fb-437e-986f-c2d0d00293e7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.527951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.529688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 219/449] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/projects/447338b8480048d8be1a3dac45927add => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:26.535285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.564178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.564940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.565295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.565643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.627747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.632964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.634435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.636904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc588b-07e9-4a91-a2ad-4c3014fa544f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.670846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.672002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.672289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.672569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] PATH_INFO: `/v3/groups/a67bcab61c334e67bb82d63b57a5adb9` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.673742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] RBAC: Authorizing `identity:get_group(group_id=a67bcab61c334e67bb82d63b57a5adb9)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.675216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.677746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 220/450] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/groups/a67bcab61c334e67bb82d63b57a5adb9 => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:26.683554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.716458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:26.732830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 231/451] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:26.738407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.784941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8bc4-dfdb-4007-bb41-329a00da112e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.817952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.818634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.818923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.819217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.820069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] RBAC: Adding query filter params (group.id=a67bcab61c334e67bb82d63b57a5adb9, role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=447338b8480048d8be1a3dac45927add) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:26.820386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.821816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.827774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 221/452] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/role_assignments?group.id=a67bcab61c334e67bb82d63b57a5adb9&scope.project.id=447338b8480048d8be1a3dac45927add&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:26.841093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f2a05-85d2-4a9c-b798-387ce77397d6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:26.882781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:26.883719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:26.884008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:26.884294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] PATH_INFO: `/v3/projects/5852b172b7b94aa0a1ee688b7045df41` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:26.885715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] RBAC: Authorizing `identity:get_project(project_id=5852b172b7b94aa0a1ee688b7045df41)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:26.887295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:26.889122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 232/453] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/projects/5852b172b7b94aa0a1ee688b7045df41 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:26.895232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:26.927248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:27.013120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc02ca-a57e-48c4-9788-7e867c4ef3f7 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:27.048426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:27.049227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:27.049514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:27.049787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] PATH_INFO: `/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:27.050955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] RBAC: Authorizing `identity:get_group(group_id=88ade5a440b14a90a70f68efdc5fb5d2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:27.052805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e173de-1ba1-4d03-b962-f09bf8c9b75f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:27.054411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 222/454] 10.222.0.13 () {56 vars in 1180 bytes} [Thu Jun 30 04:02:26 2022] GET /identity/v3/groups/88ade5a440b14a90a70f68efdc5fb5d2 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:27.060454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:27.116030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:27.161128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ef53c-4ca0-4130-bef1-5ea96ba23054 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:27.197631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:27.198287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:27.198573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:27.198852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:27.199724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] RBAC: Adding query filter params (group.id=88ade5a440b14a90a70f68efdc5fb5d2, role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=5852b172b7b94aa0a1ee688b7045df41) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:27.200042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:27.201487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79785b2d-e4c3-418c-93ac-0957c0311bac None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:27.208177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 233/455] 10.222.0.13 () {56 vars in 1376 bytes} [Thu Jun 30 04:02:27 2022] GET /identity/v3/role_assignments?group.id=88ade5a440b14a90a70f68efdc5fb5d2&scope.project.id=5852b172b7b94aa0a1ee688b7045df41&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:29.810747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a456-9b05-4e9f-8683-735c98bdf378 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:29.811821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a456-9b05-4e9f-8683-735c98bdf378 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:29.812302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830a456-9b05-4e9f-8683-735c98bdf378 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:29.814138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 223/456] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:29.820160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:29.821015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:29.821488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:29.843279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:29.844806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:29.927727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d113a5-eae4-4ccc-84b1-d5198f116af3 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:29.928614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d113a5-eae4-4ccc-84b1-d5198f116af3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:29.929312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d113a5-eae4-4ccc-84b1-d5198f116af3 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:29.931724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 224/457] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:02:29.938590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:29.939469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:29.940282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:29.951501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 234/458] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:29.965292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:29.966511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:30.004781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.093874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 225/459] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:30.101752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.136080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.164285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b8895-4e83-4740-baff-b21f4d37244f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.184513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.214383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.215232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.215517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.215795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.240522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.242340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c3f75d38-5e05-4a00-8678-43cd217eab73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:30.262356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:30.263290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:30.264158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:30.264158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:30.264158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.264158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:30.266448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 235/460] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:30.267994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16539da-6ab2-4a86-8b76-ac19f2e00fa4 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.268497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16539da-6ab2-4a86-8b76-ac19f2e00fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.268965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16539da-6ab2-4a86-8b76-ac19f2e00fa4 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.271028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 236/461] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:30.276856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114663b2-830c-40f8-9715-a6497452be8e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.277151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114663b2-830c-40f8-9715-a6497452be8e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.277434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114663b2-830c-40f8-9715-a6497452be8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.287168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb6553-93b6-42b3-b868-3980e59938e9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.294281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-114663b2-830c-40f8-9715-a6497452be8e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:30.295869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-114663b2-830c-40f8-9715-a6497452be8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:30.335170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.336430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.336909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.337372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.349371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.352060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.367140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 237/462] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:30.368092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114663b2-830c-40f8-9715-a6497452be8e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:30.385651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:30.386550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:30.387457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:30.387457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:30.387457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.387457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:30.389837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 226/463] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:30.399143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.401034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.487949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.492159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114663b2-830c-40f8-9715-a6497452be8e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.514314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef99f-126f-4c2c-9a04-d3d9cb1038e0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.528813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.529871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.530349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.530817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.531783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:30.532310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.533905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.546683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 238/464] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:30.547849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.552555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538292-470c-4519-be49-0b282a555c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.552555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538292-470c-4519-be49-0b282a555c0a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.552909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538292-470c-4519-be49-0b282a555c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.553173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538292-470c-4519-be49-0b282a555c0a None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.553930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538292-470c-4519-be49-0b282a555c0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:30.554244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538292-470c-4519-be49-0b282a555c0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.555684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65538292-470c-4519-be49-0b282a555c0a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.574339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 227/465] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:30.575422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538292-470c-4519-be49-0b282a555c0a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.656583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.667168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b1d7d-bf07-45d3-9cc6-44cb4503c6a1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.678064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538292-470c-4519-be49-0b282a555c0a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.705934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.707617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.708221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.708857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.715478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.716271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.716626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.716969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.717802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.719451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.727970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.731008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.731488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:30.744959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 228/466] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:30 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:30.746304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:30.752858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:30.753797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:30.754790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:30.754790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:30.754790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:30.754790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:30.757164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 239/467] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:30.766732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:30.861836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b633869b-7b4c-4494-9dfd-cc983fdc42e5 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.903804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:30.904506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.904793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.905072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.905797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:30.907364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303ad3e-db2a-427f-aa78-f7c0bc830183 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:30.932039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 229/468] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:30 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:30.933861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d334d-e6b9-4f48-a2f1-aa3ce72bc496 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.934335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d334d-e6b9-4f48-a2f1-aa3ce72bc496 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.934831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d334d-e6b9-4f48-a2f1-aa3ce72bc496 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.936855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 230/469] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:30.943362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0ff95-253c-4b0c-80b3-7b06bba62310 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:30.946238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75275-4d11-4d43-b307-91a29d2fc654 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:30.947503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75275-4d11-4d43-b307-91a29d2fc654 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:30.947503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b75275-4d11-4d43-b307-91a29d2fc654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:30.988995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-92b75275-4d11-4d43-b307-91a29d2fc654 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:30.990829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-92b75275-4d11-4d43-b307-91a29d2fc654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:31.003526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.004354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.005156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.005156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.005714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:31.006024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.007595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d062859-5259-4047-8d97-bfc060e0c52c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.021833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 240/470] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:31.022667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b0785-33d0-4315-8374-f8ca0701f43d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.022959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b0785-33d0-4315-8374-f8ca0701f43d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.023303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b0785-33d0-4315-8374-f8ca0701f43d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.025380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 241/471] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:31.030392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b0785-33d0-4315-8374-f8ca0701f43d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.033407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.056466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.088667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 231/472] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:31.090470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.091068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.095534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.119353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:31.121575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:31.163799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b0785-33d0-4315-8374-f8ca0701f43d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.203084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.203816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.204101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.204369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.205078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.206565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.213422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 232/473] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:31.214169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.232959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 242/474] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:31 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:31.250083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.332056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.376657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037c2ea0-941b-469c-ab0b-e39491115d7b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.386078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.408859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f228ea-acc4-45da-874b-49286f21f399 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.431452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.432612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.433213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.433582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.451731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.453572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.464110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.464603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.464892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.465172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.465936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.467465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:31.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:31.478771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:31.479715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:31.479715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:31.479715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:31.479715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:31.480702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.sql.core [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 04:02:31.480702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 04:02:31.480702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [parameters: {'id': '0d3295fcbd7d45cb96fc5fa69d4daa47', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 04:02:31.480702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80702) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 04:02:31.482334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 233/475] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:31.491167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application result.read() Jun 30 04:02:31.502097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:02:31.502989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application yield resource Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise self.value Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application session.commit() Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 04:02:31.503976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self.session.flush() Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application compat.raise_( Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 04:02:31.504900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:02:31.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application result.read() Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:02:31.506728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [parameters: {'id': '0d3295fcbd7d45cb96fc5fa69d4daa47', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:31.507936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 04:02:31.508729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.509551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 243/476] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 30 04:02:31 2022] POST /identity/v3/roles => generated 144 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:31.512261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.647207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86904a0-0600-469c-a75a-f1cf2030223a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.654848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76aaa9d-73c3-4de3-999b-bd3236516699 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.680181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.694401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.695116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.695471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.695818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.696221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.696689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:31.697064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.697589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.698035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.698482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.698729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.710233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:31.712875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 234/477] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:31.718481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:31.719412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:31.720433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 04:02:31.721212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:31.723773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 244/478] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:31.729449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:31.745664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:31.850668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb96b7-b658-4c18-ae3c-481e612dbf1b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.889331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.890315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.891225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.891676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.892595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:31.893095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.895519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd440ab3-c9db-4ae1-8893-db9a5e2e0e3e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:31.905752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 245/479] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:31.931707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e55b12-2584-49fd-8874-7d15c3f609ff None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:31.985264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:31.985960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:31.986248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:31.986528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:31.987284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:31.988910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f54084-7f94-41f1-ba17-6c5a210aa35e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:32.025874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 235/480] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:31 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:02:33.329199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd622e4-4890-4e1a-93dd-20cb21aa82c5 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.329726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd622e4-4890-4e1a-93dd-20cb21aa82c5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.329798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd622e4-4890-4e1a-93dd-20cb21aa82c5 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.331429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 246/481] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:33.337645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.338538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.339015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.357473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:33.359087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:33.458110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 236/482] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:33.479626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ead2e7-2b5f-4c73-a068-b2c7e91d19d7 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.480192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ead2e7-2b5f-4c73-a068-b2c7e91d19d7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.480691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ead2e7-2b5f-4c73-a068-b2c7e91d19d7 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.482260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 247/483] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:33.485531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:33.487255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.487901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.488424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.489241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:33.516298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:33.517494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:33.590057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d50b2-09ce-43c7-9991-c3708c1edced None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:33.625447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:33.626499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.626921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.627402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.639012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:33.647710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:33.648627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:33.649415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:33.651450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:33.656239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 237/484] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:33.658574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:33.714527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 248/485] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:33.740577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:33.750508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:33.807029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc3eb77-cc69-4310-a83a-ff991c1345fb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:33.861300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:33.861893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.862178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.862453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:33.865581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:33.865898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:33.867328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:33.886506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 238/486] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:33.891732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:33.911722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a089e08-3830-451e-9b8c-5d2e7be71073 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:33.978488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:33.979338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:33.979622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:33.979900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.001409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:34.009364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:34.010272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:34.011060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 249/487] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:34.024164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.071325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a386790-a653-406c-a4f9-4cad92f1dce4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.087299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.101989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.113286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.114766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.115056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.115380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.126825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.128358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-50e86bfd-085c-4783-8171-27326081bb10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:34.147873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:34.148810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:34.149665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:34.149665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:34.149665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:34.149665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:34.153710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 239/488] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:34.155034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e81e1-5bc3-4e5b-9633-6077b8047b72 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.155343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e81e1-5bc3-4e5b-9633-6077b8047b72 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.155620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7e81e1-5bc3-4e5b-9633-6077b8047b72 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.161924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 240/489] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:34.162729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e81e1-5bc3-4e5b-9633-6077b8047b72 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.169477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe7d98-f8fe-4718-9864-4595fc8a145c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.213477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.215022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.215750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.216192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.217412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:34.218286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.219947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39a281-21ef-4948-9af1-d7d3305454d8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.238010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 250/490] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:34.238010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.238010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.238010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.257542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:34.258764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:34.283064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7e81e1-5bc3-4e5b-9633-6077b8047b72 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.328298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.328974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.329257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.329536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.330260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:34.330569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.333371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.346784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 251/491] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:34.348440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.350425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 241/492] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:34.360112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.382212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.422610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.461626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad746a7-eabd-4490-822f-26d4d27c6e8d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.491888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f13702-5366-4146-ba7d-8676be64888a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.498030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.498843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.499146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.499419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.511873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.513348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.528237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.529041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.529325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.529600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:34.531393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:34.532399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:34.533428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:34.533428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:34.533428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:34.533428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:34.537525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 252/493] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:34.538404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.539566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.541156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-b1828b50-7cf5-45f4-bd96-061fdcdd76d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:34.557978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:34.559140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:34.560222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:34.562129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 242/494] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:34.563608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9087e4a-caa4-445f-909f-4189fe05de80 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.564058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9087e4a-caa4-445f-909f-4189fe05de80 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.564452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9087e4a-caa4-445f-909f-4189fe05de80 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.565851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 243/495] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:34.566694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9087e4a-caa4-445f-909f-4189fe05de80 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.639811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.661652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226e15d-247c-44c1-b060-04cabd19aa73 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.671255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.677347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9087e4a-caa4-445f-909f-4189fe05de80 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.697384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.698204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.698493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.698776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.708918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:34.714138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.714856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.715184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.715463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.716193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:34.716499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:34.716743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:34.717636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:34.718447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:34.719060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c86fece-4a5f-49b1-a58a-803ac3fab916 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.719158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 253/496] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:34.719158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.729476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 244/497] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:34.730716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d8019-f97f-4590-9857-e8bb05b7df3a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.731001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d8019-f97f-4590-9857-e8bb05b7df3a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.731307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d8019-f97f-4590-9857-e8bb05b7df3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.747180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5a7d8019-f97f-4590-9857-e8bb05b7df3a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:34.748395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5a7d8019-f97f-4590-9857-e8bb05b7df3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:34.815949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 245/498] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:34.817459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b664-b8c9-4cc8-b524-30b345c5cc49 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.817751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b664-b8c9-4cc8-b524-30b345c5cc49 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.818034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1b664-b8c9-4cc8-b524-30b345c5cc49 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.819387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 246/499] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:34.820070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1b664-b8c9-4cc8-b524-30b345c5cc49 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.833517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab24ce05-94f7-4c34-82cc-53d632b1de26 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.867035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.867745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.868029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.868307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.869147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:34.869458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.870901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.888438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.908502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 254/500] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:34.909256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:34.919212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:34.933531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1b664-b8c9-4cc8-b524-30b345c5cc49 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:34.967997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:34.968660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.968931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.969208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.969966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:34.970277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:34.971713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316efa1-91fd-4692-9884-84d98f4bd91d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:34.980223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 247/501] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:34.981386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:34.981684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:34.981966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:34.997479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:34.998689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:35.014420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446bb1d-0054-4589-9637-ca6321623bc8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.047032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.047862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.048147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.048425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.056129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.057599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.064112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 248/502] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:35.064944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:35.073768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:35.074780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:35.075726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 255/503] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:35.076279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.139347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.149121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.169703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff8c92-8fc9-4a85-a7e4-46718d6f9b90 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.184168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3eb1b-4239-4758-ae44-d49d5331130e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.203325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.204119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.204401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.204669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.214611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.217681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.218329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.218618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.218889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:35.233499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:35.234367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:35.235282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:35.235282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:35.235282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.235282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:35.235282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 249/504] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:35.235562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.285809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.333199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb36b7-a9e4-407c-be64-c11ef33a54ec None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.342320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.361729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.367274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.368080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.368360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.368640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.378502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.380167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.386171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 256/505] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:35.387039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:35.402993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:35.403982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:35.404849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:35.404849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:35.404849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.404849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:35.404849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 250/506] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:35.405114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.488200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c534a-3c75-480a-93b8-391c6ce0f0a9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.504102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad07520-c68c-4140-9e9a-46530e352d64 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.521468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.522154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.522443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.522784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.524262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:35.524580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.526033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.544001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 257/507] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:35.544825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.553402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.554231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.554514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.554794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.566689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:35.567588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:35.574931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:35.576025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:35.576995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 251/508] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:35.578804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.625434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.656806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26adaa-5e02-4bd1-83a2-24eaf8b6105e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.679978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ab4a7-ff22-4e62-a30c-5a582105dbea None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.691324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.692084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.692373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.692652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.693370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:35.693681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.695072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.706255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 258/509] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:35.707051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.717957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.718766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.719055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.719358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.729933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.731448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:35.748111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:35.748965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:35.749839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:35.749839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:35.749839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:35.749839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:35.749839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 252/510] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:35.750111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.786322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.812124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb2b2e-2d40-4b33-ad6c-7a86087cac91 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.845780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:35.847846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.848777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.849206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.849541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.850371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:35.850669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.852240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.853740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ea909-69da-4d1a-b42e-b2cee54b9b55 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:35.866635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 259/511] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:35.867529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:35.893308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:35.893971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:35.894272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:35.894635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:35.966027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:35.967508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:35.974212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227d3c2-f118-45b3-b4ed-798e6d728509 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.007138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.007736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.008012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.008296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.009036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.009340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.010725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.012302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.020457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 260/512] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.021242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.063692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.072571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 253/513] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:36.073812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.123846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7618fca-f334-4407-b86a-f6d9ac032bcd None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.155981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.156656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.156937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.157211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.157912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.159380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.172621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de6e00-da9c-424d-9014-1533dd9372d9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.181927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 261/514] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:36 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:36.182782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.214651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.215378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.215666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.215939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.216681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.216987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.218435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.231482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 254/515] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.232376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.253518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.297780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.311524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c291fd-9369-452b-a301-d09810cdb49c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.348912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.349549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f814d2-a350-47d8-a0b3-42d27478b02d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.349752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.350011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.350288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.358765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.360193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:36.374548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:36.375544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:36.376441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:36.377336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 262/516] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:36.377502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.393072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.393881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.394165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.394438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.406674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.408362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:36.426367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:36.427649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:36.429091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:36.429091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:36.429091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:36.429091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:36.429091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 255/517] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:36.429373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.487454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88566c3e-5385-4761-ae6b-086cca582612 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.500721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.527082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.527940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.528228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.528509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.539848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.541337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.544611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a9b92-aad5-43df-ada7-ca52f232068a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.554826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:36.559248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:36.565156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 263/518] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:36.566273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.596970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.598062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.598350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.598627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.606670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.608493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:36.624023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:36.624909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:36.625782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:36.629982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 256/519] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:36.630845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.671149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d61ed5-a996-407c-896f-b0242020b235 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.706306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.707198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.707492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.707770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.708622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.708937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.710664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.733885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 264/520] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.734780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.742578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba6aea-fcac-498a-970b-b01843862b66 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.760499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.778293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.778964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.779278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.779561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.780316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.780622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.782004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.793615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 257/521] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.794481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.823033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:36.845705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ed937-7694-49ef-ac98-e0b7f0c64b04 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.883661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.884572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.885025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.885442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.886318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.886765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.888338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.894997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d24cf-9678-4081-8047-77b16b313ab4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:36.904369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 265/522] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.904369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.934553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:36.935537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:36.936018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:36.936461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:36.937416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:36.937897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:36.939574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:36.957775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 258/523] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:36.959140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:36.989672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.010913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86184958-a1db-44c5-8c07-e3cbb48edb16 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.045678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.046467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.046755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.047044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.053427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.066985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b29e7-8448-491b-9536-c6dc7fbc82bc None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.104291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.105023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.105312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.105588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.112659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.114437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.116288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.117749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:37.129192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:37.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 259/524] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:37.132062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.207437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.234688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 266/525] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:37.234688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.241986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabcc06-0e4d-443e-b4f1-9180af06b743 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.284033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.284848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.285256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.285544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.293289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.294753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:37.309517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:37.310448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:37.311435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:37.315643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 260/526] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:37.316307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.330405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.344233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea886f-e8d1-4844-a40d-a2515e0c6aac None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.380427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.381017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.381298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.381567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.382402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:37.382707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.384203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.390262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 267/527] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:37.391332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.443654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3aa2-db05-4246-b167-e305e52910b1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.447857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.496889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.497999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.498559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.498978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.500288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:37.501198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.503365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.510466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf76488-2a1f-46e7-bc76-5afdebf42b23 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.549290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.550311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.550686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.551084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.556709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 261/528] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:37.557582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.569453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.570974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:37.594071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:37.595003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:37.595897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:37.595897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:37.595897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:37.595897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:37.598438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 268/529] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:37.599588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.650037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.694936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c96fd7-0024-4beb-b948-47ed21b1cd9b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.708612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761fe6-c902-4257-bcd0-023dcbdf143d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.727553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.742898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.743621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.744071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.744351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.745149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:37.745460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.746913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.747313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.747980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.748329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.748780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.763468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 262/530] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:37.764378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.830086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.831800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.895453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ce077-4879-4005-b0ab-e618b54bec3f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:37.925383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.934557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:37.935420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:37.935808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:37.936186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:37.937021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:37.937429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:37.939044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:37.955273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 263/531] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:37.956147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:37.968328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:37.975942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 269/532] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:37.976941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.070083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe6998-1574-4bd4-9a0c-4c99b2d2b611 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.101024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3526c-a9e3-43c7-baa0-4006f00e88c2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.108539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.109209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.109497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.109779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.110617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=aecdfa0fc8514b09b04bd7ee759d21ba) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:38.110938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.112416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.118986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 264/533] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/role_assignments?user.id=aecdfa0fc8514b09b04bd7ee759d21ba&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:38.119897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.164636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.165468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.165747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.166023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.176390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.187607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.189146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:38.213924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:38.216105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:38.218183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 270/534] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:38.227138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.232420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3ecf-d1c0-4ace-92d3-db9dc8c4258b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.271294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.272181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.272479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.272758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.284586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.286227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.290405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:38.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:38.307988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:38.308825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:38.308825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:38.308825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:38.308825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:38.308825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 265/535] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:38.314241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.342216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfc392-7010-4ce4-b599-8b0d9e2e689e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.381774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.382853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.383369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.383861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.384871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:38.385393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.387046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b033e-2e88-4c8e-8a02-9edd15c99284 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.406354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 271/536] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:38.408024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60d30-53c7-415e-afd7-3815ad72d0b3 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.408524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60d30-53c7-415e-afd7-3815ad72d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.408989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b60d30-53c7-415e-afd7-3815ad72d0b3 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.411321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 272/537] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:38.417536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.418103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.418614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.428674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce30fc-3da8-42d3-92b7-68ce4d9ec0cb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.431760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.437712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:38.439133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:38.479120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.479120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.479120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.479120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.479843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:38.479843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.480535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.496429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 266/538] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:38.497304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.521345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 273/539] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:38.522399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.591346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.651479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87562ccc-45f5-4cd8-bfee-4720f04266fd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.653879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af536b43-d17f-4d94-958b-e44b94c45cea None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.695635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.696605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.696961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.697343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.697776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.698322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=d534d56b90834341afb915c39e70faff) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:38.698722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.699051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.699487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.699891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.700464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.711130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 267/540] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/role_assignments?user.id=d534d56b90834341afb915c39e70faff&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:38.711130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.715125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.716378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:38.743785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:38.744718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:38.745562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 274/541] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:38.750799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.773111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.865677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d21fc-b90d-46c7-9674-91ad0a3034c9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.875816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:38.897959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444a142-5f91-4119-b648-e8d2d4fd0231 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:38.905593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.906291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.906578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.906858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.907836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:38.908146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:38.909787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:38.924111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 275/542] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:38.930378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:38.961570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:38.962408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:38.962690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:38.962964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:38.993191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:39.006201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:39.008690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:39.010553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 268/543] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:39.017969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:39.071952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0114-5d32-4705-a310-02cdaf8a5e24 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:39.107953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:39.108632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.108926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.109205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.110219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=00a87fd4c488404f98aa12190f598cfe) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:39.110542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:39.112014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:39.113827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:39.118054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 276/544] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:38 2022] GET /identity/v3/role_assignments?user.id=00a87fd4c488404f98aa12190f598cfe&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:39.131281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc9514-ef68-41ef-90a7-e3aa1d3566b3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:39.186375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:39.187503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.188006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.188537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.189557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:39.190112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:39.191892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:39.208085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 269/545] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:39.214106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:39.264563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:39.361103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1202979-3ccc-4df6-ab94-80eb2c3c0048 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:39.401236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:39.402070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.402354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.402628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.416783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:39.418272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ac9fbb47-dbcc-4095-ab19-ff803faf47de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:39.442744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:39.447890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:39.448732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:39.448732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:39.448732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:39.448732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:39.450699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 277/546] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:39.460237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:39.634256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8476-ed31-4551-9286-278a8b4b792c None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.634773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8476-ed31-4551-9286-278a8b4b792c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.634844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea8476-ed31-4551-9286-278a8b4b792c None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.636452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 278/547] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:39.641073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5d3b0-6782-43a1-8dc9-7e9e9885a386 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:39.641501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.641765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.642045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.672817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:39.674034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:39.719210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:39.719891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.720180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.720463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:39.721210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:39.721520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:39.723123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:39.753007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 270/548] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:39.754087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:39.777236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:39.799601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 279/549] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:39.823929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:39.852078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:39.933464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddad0a8-93d4-464e-8db6-d4296e02b52d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:39.996333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:39.997150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:39.997432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:39.997703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.006634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8322f6cd-c576-4369-aa74-1114c29bc9d9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.013353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:40.042348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:40.043836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:40.044883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 271/550] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:40.053259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.071229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.072413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.072898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.073369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.085353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:40.094471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:40.095437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:40.096277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:40.099313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 280/551] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:40.104757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.188815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.248530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.264011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5f89f-cc4b-4710-a7a5-af89bb628eaa None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.301096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbe377-4553-403c-9574-2e40c0982e09 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.320879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.321986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.322499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.322974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.324042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:40.324649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.326293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.349558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 272/552] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:40.351142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.376744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.377444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.377737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.378015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.378780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:40.379545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.380968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.394821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 281/553] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:40.400331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.461280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c02622-6af6-4493-b6a6-ca3b4ce97896 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.489064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.499940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.500933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.501439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.501913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.579175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.584171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3df482-07ae-463f-a2fa-a7b34e785b7f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.589219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642217d9-862b-45d0-9974-5a3eaa07a483 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.686252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.687006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.687315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.687598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.690499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.696088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 273/554] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 04:02:40.698118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d478-1767-4204-9127-dcd0e1540b16 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.699541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d478-1767-4204-9127-dcd0e1540b16 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.699541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648d478-1767-4204-9127-dcd0e1540b16 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.699776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.701225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.701442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 274/555] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:40.702312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648d478-1767-4204-9127-dcd0e1540b16 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f1f1b861-b63f-450f-9df3-5a67de3fc279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:40.724751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:40.725716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:40.727202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:40.727202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:40.727202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:40.727202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:40.727202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 282/556] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:40.727491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.727787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.728068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.736712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.748834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:40.750046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:40.823801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648d478-1767-4204-9127-dcd0e1540b16 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.827532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 283/557] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:40.828435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.860309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.861540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.862018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.862502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.873639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.875899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:40.895752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:40.897173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:40.898069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:40.898069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:40.898069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:40.898069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:40.904836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 275/558] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:40.906122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:40.953089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7439a-c1d7-4af8-90fb-79d632816697 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:40.972143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:40.990750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:40.991482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:40.991813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:40.992110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:40.992982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:40.993296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:40.994805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:40.995597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.009433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 284/559] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:41.010073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.019940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb709c3-339c-4028-a620-482591d851a0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.057438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.058669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.059279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.059758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.071659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:41.080607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.081624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:41.082537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.085505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 276/560] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.086618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.122154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1654c-4d02-4eb9-ae8a-aa2f3479e7a4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.158708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.159420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.159726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.159997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.160850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:41.161159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.162685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.177788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 285/561] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:41.177972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.200608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246dead4-7fea-4876-a623-b280723c0daa None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.220387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.241482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.242291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.242608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.242860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.250733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.252235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.254169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:41.268137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.269212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 277/562] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.270229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.297577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3ef32-4a4f-4c6d-82f6-e140b7344618 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.336787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.337568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.337861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.338142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.338894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:41.339239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.340664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083fe8c-7337-48b0-b5e8-9728c0880978 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 286/563] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886a19-bccf-4f35-9a2a-2e71c2ded72c None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886a19-bccf-4f35-9a2a-2e71c2ded72c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d886a19-bccf-4f35-9a2a-2e71c2ded72c None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 287/564] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:41.353442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886a19-bccf-4f35-9a2a-2e71c2ded72c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.383630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34991-c891-4876-9872-95fe5b6f8770 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.423205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.424006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.424295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.424576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.432052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.433540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:41.448070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.448982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.449871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 278/565] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.450347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.463860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.468051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d886a19-bccf-4f35-9a2a-2e71c2ded72c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.502367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.502966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.503291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.503568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:41.504874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.506314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66306395-c7cf-420f-9758-b2610c059028 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.508824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.524071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 288/566] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:41.525301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63551-ecb3-4ce0-bea6-83a877a2fd0c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.525625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63551-ecb3-4ce0-bea6-83a877a2fd0c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.525877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d63551-ecb3-4ce0-bea6-83a877a2fd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.541967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-90d63551-ecb3-4ce0-bea6-83a877a2fd0c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:41.543194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-90d63551-ecb3-4ce0-bea6-83a877a2fd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:41.549662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6090e3-be7c-4781-94cb-a853ca495ba4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.587711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.588524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.588808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.589080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.599542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.600990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.609630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 289/567] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:41.610956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea69f9-1a02-4e7d-96df-9d5c79a37267 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.611301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea69f9-1a02-4e7d-96df-9d5c79a37267 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.611584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea69f9-1a02-4e7d-96df-9d5c79a37267 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.612835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 290/568] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:41.613617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea69f9-1a02-4e7d-96df-9d5c79a37267 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:41.628135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.629066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.630063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:41.630063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:41.630063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:41.630063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.630063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 279/569] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.631921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.707142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.721236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea69f9-1a02-4e7d-96df-9d5c79a37267 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.730143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a7be5-7d05-453a-be7d-8517809445f3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.740787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.756349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.756961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.757245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.757525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.758313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:41.758627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.760107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce136e9-9d8c-4621-b681-632ea6b8fec8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.764305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.765095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.765376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.765649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.771945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 291/570] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:41.773160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.773455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.773740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.776035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:41.777497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:41.790378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:41.791612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:41.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.796334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.797229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:41.797229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:41.797229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:41.797229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.797229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 280/571] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.797513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.860858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 292/572] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:41.861694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.903187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ccc-1356-428d-ae05-b3e038467c7b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.939559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:41.940376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:41.940606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:41.940707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:41.940928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:41.952378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:41.960265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:41.961237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:41.962079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 281/573] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:41.962774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:41.962842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa805132-769d-4dfc-a118-be48590fa4b6 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:41.990071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.000987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.001674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.001958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.002232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.002968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.003312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.004817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.016846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 293/574] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.017652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.064913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec509e3-fb61-4fa1-b0b7-d489f0e40c5d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.098792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.099507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.099794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.100072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.100918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=0857fc0c4021456b88288ba10921ca6d) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.101234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.102701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.108834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 282/575] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:41 2022] GET /identity/v3/role_assignments?user.id=0857fc0c4021456b88288ba10921ca6d&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.109431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.116680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9507e250-4015-45ad-bb82-507f0c226041 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.156629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.157626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.157626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.157897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.158612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.158924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.160500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.169325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.180913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 294/576] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.180913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.216579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.234485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f131-7ba1-412e-a32b-1ffcc606c02a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.273901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.274640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.274921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.275229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.288081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:42.295967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:42.296890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:42.297727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:42.298638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67658da-c786-4a1d-9980-0e035cdd0cac None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.298841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 283/577] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:42.299348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.332280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.333067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.333415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.333761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.334588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.334969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.336442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.344857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 295/578] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.345671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.408904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.413451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be0a5b-5e75-4a43-a096-3e92e0a928af None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.448956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.449679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.450198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.450314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.453108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a654f51-5ff2-46f8-9082-cfc2a5b2dc85 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.458504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.460239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:42.474843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:42.476021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:42.477175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:42.479057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 284/579] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:42.479845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.490615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.494472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.495314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.495596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.495872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.505572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.507035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:42.521694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:42.522601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:42.524242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:42.527727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 296/580] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:42.528573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.593805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15620755-787b-4c49-88ba-89ef8091b93b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.634471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.636137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.636976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.637665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.639115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.639414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba181c-885a-43df-9c63-f08325e3fb38 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.640180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.642966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.657737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 285/581] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.659338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.666067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.682404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.683243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.683531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.683810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.694629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.696190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:42.718886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:42.720319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 297/582] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:42.721225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.743792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.778568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab9b1a-ca56-4f34-b7b2-7f5131f179df None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.815040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.815763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.816046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.816367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.817574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.817887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.819367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.826517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8afdee-b516-4b76-9f57-8c25857a58e0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.838812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 286/583] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.839475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.865895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.866628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.866918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.867233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:42.868027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:42.868339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:42.869778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:42.881684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 298/584] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:42.882485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:42.916880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.949588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ef82c-83d8-4421-9fcd-e068d9464bf8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.975744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:42.985998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c43691-02d1-48d7-8a79-417a2444993f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:42.988991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:42.990168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:42.990589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:42.990996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.006058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.008646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.020954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.021562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.021848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.022147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.022856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:43.023195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.024595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:43.029806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:43.030664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 287/585] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:43.037035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.048270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 299/586] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:43.048270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.145995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.147502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1cc37-b69a-4b56-a3a1-4f7a7175c3df None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.156420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8055742-916b-4036-9682-b1166ed4f099 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.182812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.183659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.183941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.184210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.194196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.194817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.195280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.195634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.195977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.196330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.211445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=67b4385cb5ed42d1b9488d409abd61b1, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.212934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:43.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:43.215683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:43.216577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:43.216577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:43.216577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:43.216577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:43.220751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 288/587] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:43.221592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.229026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 300/588] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:43 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:43.229880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.276985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.352728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bcce8-9e44-4f37-a8f5-3f2815c33d0a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.376970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173ae69-79c5-466c-8349-8743efac7cc9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.401542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.402717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.403005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.403426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.411598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.413155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.419583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.420473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.420931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.421389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.422189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:43.422572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.424034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:43.430607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:43.431820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:43.432749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:43.433990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 301/589] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:43.434911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.440092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 289/590] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:43.441487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.442197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.559616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109949-d53f-4cfb-a9f4-aa29f7f76a85 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.566470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.576701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b5dd0-f821-421c-ba38-ce6ad4a04905 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.605580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.606647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.607192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.607667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.608681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:43.609271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.610902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.621745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.622576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.622995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.623309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.624167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:43.624481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.625914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.626145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 302/591] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:43.627209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.659401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 290/592] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:43.660798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.712617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.764822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418fce13-61f0-4f78-8944-d1e624236c82 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.767396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba63a0-8618-4d74-a460-95b1b5464d6c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.809626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.810527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.810812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.811090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.811572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:43.812615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:43.813104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:43.813574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:43.819351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.821111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.821817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:43.823498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:43.836783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:43.837748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:43.839319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 291/593] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:43.842677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:43.843587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:43.844525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:43.845052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.846850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 303/594] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:43.848263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:43.857053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:43.962646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e4666-68bb-488a-8e63-aa891dd5e346 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.978278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67ab73-011c-4a32-a8e0-1bb43884d069 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:43.992072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.006722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.007810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.008362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.008930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.009925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:44.010449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.012317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.019963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.021360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.021648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.021930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.026433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 304/595] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:44.027560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.038692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.040809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:44.059980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:44.060936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:44.061842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:44.061842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:44.061842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:44.061842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:44.064308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 292/596] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:44.065334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.140623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.146421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2390b-610f-406d-b58d-2164debbb6db None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.182342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.183027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.183330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.183602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.184443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:44.184750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.185320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e043bf-fd79-475d-b963-3a3464301408 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.186181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.204280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 305/597] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:44.204900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.225914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.226816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.227139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.227435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/aecdfa0fc8514b09b04bd7ee759d21ba/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.242988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=aecdfa0fc8514b09b04bd7ee759d21ba, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.244579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676b617e-76c5-4687-b228-02c7637c7405 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.245836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.261756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 293/598] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:44 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/aecdfa0fc8514b09b04bd7ee759d21ba/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:44.263062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9af-b7dd-4db1-9f18-964bb8ea6a9a None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.263396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9af-b7dd-4db1-9f18-964bb8ea6a9a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.263671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1b9af-b7dd-4db1-9f18-964bb8ea6a9a None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.265122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 294/599] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:44.270286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b9af-b7dd-4db1-9f18-964bb8ea6a9a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.337898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d960b-49eb-4efe-9f51-2835e1ba58d5 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.382634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.384631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.385265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.385745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.387698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:44.388299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.390476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b456e48-1548-4fcb-a887-e1647b615154 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.408606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 306/600] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:44.408867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1b9af-b7dd-4db1-9f18-964bb8ea6a9a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.410899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.411410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.412931 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.422542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.436809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:44.438199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:44.464652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.465464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.465745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.466026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.477089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.478547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:44.502629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:44.503606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:44.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:44.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:44.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:44.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:44.504562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 295/601] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:44.505881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.522119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 307/602] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:44.523222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.561112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.613945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58f10d-b9b8-47b2-a475-8ff7efef1185 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.649751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.650661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb035-d36e-480c-bd81-ce477010d939 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.651206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.651703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.652174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.659581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.661572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:44.676191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:44.677458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:44.678374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:44.680972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 296/603] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:44.685963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.689951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.690639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.690985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.691366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.692692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:44.692692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.693843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.706410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 308/604] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:44.707495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.714711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.794459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccfb54-382f-4228-b731-ac6b26333d47 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.805813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.815619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476328a8-6a67-46fb-bf16-6e045e9de5b6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.831254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.832283 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.832606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.832889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.843742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:44.851015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:44.851731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:44.852019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:44.852203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:44.853100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:44.853934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:44.854568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:44.854568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:44.854568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:44.854758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 297/605] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:44.854758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.854885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:44.868221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 309/606] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:44.869352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:44.946348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:44.966960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8333f0e-03c2-4b43-8d33-7c7f7085b576 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:44.980612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8434d-e236-4ad1-a323-09130c7f4957 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.002733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.003983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.004447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.004902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.012139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.013801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.018073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.018765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.019053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.019484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.020253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:45.020558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.021945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:45.030823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:45.031794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:45.032762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:45.033184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 310/607] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:45.033184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.035123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 298/608] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:45.045662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.050614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.149216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2efec4-9893-4bf1-8ca8-5663bc664269 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.159625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d99bd9-56bc-44a4-8565-377be963872e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.193537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.194955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.195548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.195964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/d534d56b90834341afb915c39e70faff/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.202314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.203043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.203470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.203848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.204751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:45.205161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.206727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.214566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=d534d56b90834341afb915c39e70faff, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.216687 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.227281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 299/609] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:45.227281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.232105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.239497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 311/610] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:45 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/d534d56b90834341afb915c39e70faff/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:45.240353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.321005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.370571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f751272-5fc9-4f57-a008-aefd70f68241 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.386346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac8681-a3d5-417a-b038-091d2b44115d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.411069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.411803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.412090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.412367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.427937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.429449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.433636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.434895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.435450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.435920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/00a87fd4c488404f98aa12190f598cfe/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-78efe979-47fc-46a1-97eb-686ea68d6be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:45.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:45.450286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:45.451242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:45.451242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:45.451242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:45.451242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:45.451908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 300/611] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:45.452963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1459-dcd4-474f-9d42-1b187f32e8f5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.453064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=00a87fd4c488404f98aa12190f598cfe, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.453226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1459-dcd4-474f-9d42-1b187f32e8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.453477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d1459-dcd4-474f-9d42-1b187f32e8f5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.454411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336015ac-4e3e-4c6a-b19e-cee5c149666c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.454892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 301/612] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:45.458309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d1459-dcd4-474f-9d42-1b187f32e8f5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.471784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 312/613] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:45 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/00a87fd4c488404f98aa12190f598cfe/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:45.474055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.474351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.474633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.495671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:45.497737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:45.527163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.611515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 313/614] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:45.631406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d1459-dcd4-474f-9d42-1b187f32e8f5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.633941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.691021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.719228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.720050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.720398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.720745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.722115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:45.723311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.726698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.745058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 302/615] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:45.750256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.778577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739bb72-4783-4241-88dd-47eba0d62a6e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.816249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.818251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.818712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.819283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.830682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:45.839642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:45.840599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:45.841435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:45.847294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 314/616] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:45.850716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.876316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bbe55-6e70-4357-9bd8-706625d39bb2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:45.893219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:45.916250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:45.917440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:45.917918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:45.918396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:45.926953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:45.928757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:45.944405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:45.945395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:45.946358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:45.949206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 303/617] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:45.955242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:45.995301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddc7e9-6f8c-4812-af5c-e3ec87b4d7df None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.000702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.065698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.067256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.068751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.071974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.072947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:46.073408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.074967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.090342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 315/618] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:46.097644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.159665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4a424-1b50-406f-aeda-b728c0603729 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.225006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.226094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.226384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.226654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.228535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:46.228849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.230298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.262555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 304/619] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:02:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:46.267621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.294652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a88e17-480d-4a05-8386-23a601f86597 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.325624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.370205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.371030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.371339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.371619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.396013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.398036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:46.423239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:46.424556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:46.425796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:46.425796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:46.425796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:46.425796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:46.425796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 316/620] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:46.431998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.461385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207c4e3-735e-4d44-a19e-456360a3f235 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.471620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.526014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.526737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.527071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.527387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.538326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.541080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-028b71c1-bc82-431a-98a3-058ae3e8a108 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:46.563491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:46.564596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:46.565714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:46.565714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:46.565714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:46.565714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:46.570310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 305/621] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:46.571762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b115-b04e-4d7c-91f7-621e422c9242 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.572048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b115-b04e-4d7c-91f7-621e422c9242 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.572335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6b115-b04e-4d7c-91f7-621e422c9242 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.574447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 306/622] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:46.575282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6b115-b04e-4d7c-91f7-621e422c9242 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.595825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2beef17-24ad-46ab-9ee5-aad944b72ca9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.637889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.638885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.639209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.639494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.640228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:46.640538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.642704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a057-d131-44f9-85f3-e26612d4555d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.658326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 317/623] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:46.660685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.660980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.661262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.676985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.683270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:46.685613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:46.696381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6b115-b04e-4d7c-91f7-621e422c9242 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.736575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560446-f47f-461e-954f-982649014559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.737250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560446-f47f-461e-954f-982649014559 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.737535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560446-f47f-461e-954f-982649014559 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.737810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5560446-f47f-461e-954f-982649014559 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.738728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5560446-f47f-461e-954f-982649014559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:46.739036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5560446-f47f-461e-954f-982649014559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.740561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5560446-f47f-461e-954f-982649014559 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.752974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 307/624] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:46.753757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560446-f47f-461e-954f-982649014559 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.775439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 318/625] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:46.776568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.785862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.864784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5560446-f47f-461e-954f-982649014559 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.901844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.902606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad35294-fa7b-4a64-90e9-1677485cff0f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:46.903807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.904286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.904786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.914041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.915791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:46.931606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:46.932751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:46.933946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:46.936310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 308/626] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:46.937460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:46.937925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:46.938734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:46.939018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:46.939337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:46.942066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:46.947761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:46.949499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:46.965342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:46.966444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:46.967504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:46.974325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 319/627] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:46.975367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.041100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1a028a-97fe-4524-b702-27be39d74cc0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.042968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.078189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.079001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.079311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.079589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.080565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6598b-07f5-4316-ae82-8bf4156f20c2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.090189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:47.098855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:47.099890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:47.100872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:47.105043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 309/628] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:47.105978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.118886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.119906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.120344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.120763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.121739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:47.122277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.124028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.142654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 320/629] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:02:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:47.143778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.170891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.211515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533bb6-4052-4727-85ed-4ec3c8ea2bd0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.246098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75620b-87a7-4074-8a7f-f39299bf4339 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.247136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.247855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.248206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.248486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.249840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:47.249840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.251146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.268945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 310/630] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:47.269943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.274824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.286785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.287823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.288328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.288735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.289670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:47.290139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.291979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.302102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 321/631] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:47.303125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.381281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac8819-b155-4ced-a1cd-76ecf979bb12 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.406263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.413757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b7a3b-4dd9-4cf6-9018-44288dc4902e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.419787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.420623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.421049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.421485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.455806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.456720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.457260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.457592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/0857fc0c4021456b88288ba10921ca6d/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.476995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=0857fc0c4021456b88288ba10921ca6d, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.478543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.507324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 322/632] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:02:47 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/0857fc0c4021456b88288ba10921ca6d/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:02:47.509386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.515232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.517631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.597636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.719545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19889cab-0f84-40a9-bf8d-784dac501665 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.729915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 311/633] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:47.736268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.756311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.757131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.757417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.757691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.768437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.769801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.772439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-6eba75fd-72cf-4e01-8863-8f64f3f8de99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:47.795126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:47.796259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 323/634] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5f73-e56b-4fa3-9d64-dfa7d7baf772 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5f73-e56b-4fa3-9d64-dfa7d7baf772 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c5f73-e56b-4fa3-9d64-dfa7d7baf772 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.797329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 324/635] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:47.797949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5f73-e56b-4fa3-9d64-dfa7d7baf772 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.872042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8991981-ecf7-4b4f-987a-30ae5b653537 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.891543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:47.905913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c5f73-e56b-4fa3-9d64-dfa7d7baf772 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:47.919860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.921323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.921613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.921885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.933473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.934943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.940608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:47.941417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.941672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.941950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.942688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:47.942997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:47.944411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766eb7b1-92ef-49bd-802f-caf9cdf33a00 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e5ef50a4-cdec-4986-96f2-c1f90a3d5059 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:47.953989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:47.954877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:47.955792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:47.955792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:47.955792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:47.955792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:47.956009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 325/636] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:47.957548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.957710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.957961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.959740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 312/637] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:47.961221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff0ca-2dcb-4ce4-94b7-4c7b0a35fe3d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:47.961691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff0ca-2dcb-4ce4-94b7-4c7b0a35fe3d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:47.962215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff0ca-2dcb-4ce4-94b7-4c7b0a35fe3d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:47.963714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 313/638] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:47.964665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dff0ca-2dcb-4ce4-94b7-4c7b0a35fe3d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:47.980525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:47.981725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:48.051738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.052674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 326/639] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:48.053406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.080678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dff0ca-2dcb-4ce4-94b7-4c7b0a35fe3d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.119088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.119928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.120212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.120486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.128203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.129944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.132392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-3a67ec2c-3a33-4db0-b24f-85cc39a3b8e4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:48.145265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:48.146236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 314/640] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.148632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.149364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.162015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc2ad8-8796-46e9-985f-c00075ada352 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.164636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:48.165836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:48.200133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.200819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.201110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.201444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.202185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:48.202498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.203926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.226126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 327/641] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:48.226847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.246612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 315/642] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:48.248000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.317101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.349909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe085ed-289c-4717-85e0-5547db52fb04 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.355484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b090c47-3746-4bbb-8904-865f0bba01ae None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.386566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.387339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.387621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.387895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.390221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.390935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.391255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.391600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.392462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:48.392771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.394275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.395334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.400057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:48.408507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:48.409717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 328/643] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:48.410704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.422807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 316/644] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:48.422807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.516303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4d7e7-296d-48c4-b5fe-4a03c9d27360 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.527883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d90bad-e86d-4773-8f37-5454d291a6f8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.551586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.552479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.552941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.553589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.553766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.561273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.562787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.573436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.574571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.574854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.575154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:48.581890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:48.583151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 329/645] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:48.584230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.593129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:48.601664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:48.602594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:48.603487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 317/646] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:48.604181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.660350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.708708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c47f-408b-4555-a3b6-7e11eaa573bd None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.726702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583b6c5-d1dc-488d-8d77-fe397b93f559 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.745150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.745844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.746132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.746411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.747425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:48.747554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.748925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.757291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 330/647] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:48.758094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.766488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.767185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.767476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.767758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.828798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.843922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.845441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.868517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686b89b-7404-4607-ae7a-9de9c1992e38 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:48.890412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:48.906784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:48.907497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:48.907774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:48.908054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:48.908851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:48.909169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:48.910697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:48.922678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 331/648] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:48.928754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:48.955600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 318/649] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:48.955600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.036709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9d9a9-290c-42f0-85f4-4d96af0809ac None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.043832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.061560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65075f-747f-4828-b0ff-cbef62fd0783 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.075352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.076247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.076378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.076583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.077328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:49.077638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.079038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.087815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 332/650] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:49.093078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.096944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.097908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.098435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.098435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.108835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.110291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:49.128453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:49.130003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:49.130891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:49.130891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:49.130891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.130891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:49.134192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 319/651] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:49.134995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.141075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.206171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c883826-593b-4525-9a39-c9d789d0d5b0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.236748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c22ea7-a4d1-4827-84c0-e8d0756abe04 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.245470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.246136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.246415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.246694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.247565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:49.247876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.249313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.256365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 333/652] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:49.268206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.274534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.276675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.277168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.277597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.281890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.293574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.296188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:49.321625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:49.322795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:49.323923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:49.323923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:49.323923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.323923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:49.325939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 320/653] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:49.327639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.402073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f59766-9477-4be4-90d7-5aa08d27fb8b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.410887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.440950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.441865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.442157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.442430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.444585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3595300-94ad-4296-9827-ebda0e967778 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.455299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.456973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:49.476203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:49.477170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:49.478066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:49.478066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:49.478066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:49.478066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:49.478066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 334/654] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:49.478909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.488844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.489531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.489872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.490151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.490938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:49.491284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.493155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.505904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 321/655] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:49.506748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.575307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.622815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839ccfa-e8e8-4f72-a6f4-cb2228b69754 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.646213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cf625-658e-479f-ade3-8b8dece9f578 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.661982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.662662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.662948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.663258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.663991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:49.664325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.665692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.678989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 335/656] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:49.679668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.686268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.686950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.687280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.687563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.688301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:49.688611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.689994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c101551-8db0-4cf1-bded-1045c1eeb2fd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.699715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.704475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 322/657] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:49.705789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98d47-4462-486d-ae62-6381a030af03 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.706077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98d47-4462-486d-ae62-6381a030af03 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.706356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98d47-4462-486d-ae62-6381a030af03 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.708111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 323/658] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:49.708597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98d47-4462-486d-ae62-6381a030af03 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.792414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89f569-41ab-452a-8754-391c49945302 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.823717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98d47-4462-486d-ae62-6381a030af03 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:49.827959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.828782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.829066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.829344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.835193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.837384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.838870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:49.853936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:49.855069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:49.856068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 336/659] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:49.856681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:49.872287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:49.873485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.873778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.874063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.881393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:49.882865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-2d526805-754c-4ba2-a32e-b632f2b64c28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:49.898310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:49.899348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:49.900247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 324/660] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:49.901092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:49.901245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:49.901399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:49.923319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:49.924768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:49.950050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:49.975200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d2274-c90a-4a36-a4e6-2fdd5bb0583a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.000174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 325/661] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:50.001656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.018632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.019806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.020383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.020762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.029231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.030828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:50.046163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:50.047474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:50.048483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 337/662] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:50.049187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.117532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.121617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b3d18-ba17-49df-9f3b-cc463cb4ccaa None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.155488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.156216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.156443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.156717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.157551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:50.157860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.159344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.165080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c730-a75e-46e5-b7eb-984f002e9aab None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.177815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 326/663] 10.222.0.13 () {56 vars in 1137 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:50.178941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.206814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.207870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.208342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.208793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.209767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:50.210252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.211876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.219084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.230978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 338/664] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:50.231909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.283545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417d5702-7609-4665-880e-0c8a4aa4f9b4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.320435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.321252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.321539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.321818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.332173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:50.339335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dbf1e-b568-4baa-962d-25c4c9cdc446 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:50.339894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:50.340823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:50.341681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 327/665] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:50.343072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.365107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.377939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.378674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.378960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.379287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.380135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:50.380452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.381918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.404533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 339/666] 10.222.0.13 () {56 vars in 1139 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:50.405368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.452300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dc070-732c-4c97-868a-609d87fca16a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.466581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.490789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.491410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.491691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.491964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.512757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70166-c94c-47fd-97cc-f3e5b9a28076 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.555620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.556574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.557046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.557330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.558175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=aecdfa0fc8514b09b04bd7ee759d21ba) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:50.558493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.559956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.566573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 340/667] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/role_assignments?user.id=aecdfa0fc8514b09b04bd7ee759d21ba&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:50.569110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.579522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.581028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.601699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.688516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfbf6-61cf-4995-9dc7-e41d390fab2e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.700613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 328/668] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:50.700613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.718242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.735740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.737540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.738179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.738669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.739853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:50.740367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.742048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.751549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 341/669] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:50.752702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:50.812480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc762255-23a7-42db-af57-5ba543d73f69 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.832985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:50.857950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.859035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.859476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.859812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.882376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306dd4d-6fe1-4e21-a803-806eb4a37dd9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:50.931064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:50.931967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:50.932255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:50.932536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:50.942450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.944042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:50.944176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.945507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:50.967466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:50.968893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 342/670] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:50.970209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.017555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.070384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 329/671] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:51.070384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.086595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.091719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c906fb7-8e34-453b-9c22-fbe6ebbe2cd6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.127410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.128487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.129006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.129441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.140737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.142442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:51.161051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:51.162003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:51.162922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:51.162922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:51.162922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:51.162922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:51.165690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 343/672] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:51.166751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.174293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e99f2e-f043-4dcd-9ca5-ad1fa17f2a03 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.211044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.212310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.212310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.212510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.213300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:51.213678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.215167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.231665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 330/673] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:51.231665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.282385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.298282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ef4b9-f8be-4c22-8484-63386f981333 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.322763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.335403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd56481-f792-45a2-b0ee-665e58a819a7 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.343695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.345061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.345507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.345938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.360084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.362399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.371291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.372305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.372691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.373065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.373902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:51.374310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.375976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:51.386621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:51.387747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:51.388839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:51.388839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:51.388839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:51.388839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:51.395272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 344/674] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:51.395893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 331/675] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:51.396764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.397357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.508235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1de319-bf32-4d13-82d8-1feb8417f27f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.516809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c90dd-2743-4d97-b22c-e1e857586b17 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.542821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.543963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.544292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.544585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.547476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.553913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.555621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.558254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.558831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.559130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.559424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.567649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.569520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:51.570921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:51.571893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:51.572849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:51.573393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.573464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 332/676] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:51.573464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-6e91cb67-808f-4531-9abd-ec186b5df0bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:51.592752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:51.593730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:51.595152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 345/677] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:51.595689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0599-f7c8-4a84-bcc6-71935ddd26fa None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.595909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0599-f7c8-4a84-bcc6-71935ddd26fa None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.596181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f0599-f7c8-4a84-bcc6-71935ddd26fa None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.597745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 346/678] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:51.599898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0599-f7c8-4a84-bcc6-71935ddd26fa None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.694425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55955d25-e0bb-4bed-a701-cb6ed453c327 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.712609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f0599-f7c8-4a84-bcc6-71935ddd26fa None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.729443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.730110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.730393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.730669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.731428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:51.731729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.733100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.745390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 333/679] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:51.746179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.754442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.755533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.756014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.756498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.757625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:51.758110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.759865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a873-de4f-420c-9e98-6c5d4a5fc0d6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.778823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 347/680] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:51.780013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.780302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.780579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.798804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:51.799955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:51.810199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.860598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418686a3-9c02-489a-a8c9-bff032ec053f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:51.861609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:51.880215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 348/681] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:51.880890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.918995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:51.919985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:51.920417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:51.920840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:51.922548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:51.923056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:51.924843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:51.944900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 334/682] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:51.945856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:51.990390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadfc17b-0479-4b08-a996-55517cbdafd4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.027167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.027990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.028266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.028538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.036151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.037969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:52.052481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:52.053337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:52.054247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 349/683] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:52.054749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.057601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236b13b-7ece-48cf-8d3e-6252ad84fdcd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.078617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.098458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.099146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.099459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.099715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.107120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.163189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac428777-3e92-4472-a29a-840725c47144 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.173608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.175080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.198296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.199132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.199430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.199699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.210548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:52.217997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:52.218939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:52.219813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 350/684] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:52.221435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.287558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 335/685] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:52.292939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.312777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.323992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c800c8-f1c4-4779-bfce-55ee9c11f2aa None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.331264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.361169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.361988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.362448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.362556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.363631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=d534d56b90834341afb915c39e70faff) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:52.363951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.365423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.383198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 351/686] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/role_assignments?user.id=d534d56b90834341afb915c39e70faff&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:52.383198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.413293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cccc2c-5845-4947-b063-8aa940a8aa57 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.492292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.493037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.493327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.493612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.494479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:52.494802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.496826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.519684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 336/687] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:52.520693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.583902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d823f357-57e7-4794-a50e-c535f79108bf None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.633315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.634649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.634811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.635240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.635614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.636912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:52.637499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.639190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.654578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 352/688] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:52.657000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.669089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df64935-397a-473a-8897-63fca4f3099d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.701926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.711736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.712566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.712852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.713130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.726445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.727943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:52.755811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:52.756806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:52.757725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:52.757725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:52.757725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:52.757725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:52.757725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 337/689] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:52.758450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.825529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1eaa98-6d08-4b8c-b08b-31411c22392a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.886153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.887493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.887780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.888061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.888906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=00a87fd4c488404f98aa12190f598cfe) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:52.889221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.890038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656e26-ea50-4b31-8398-c798e06e9e5b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:52.890671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:52.902820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 353/690] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/role_assignments?user.id=00a87fd4c488404f98aa12190f598cfe&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:52.904046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:52.941313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:52.958414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:52.959289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:52.959573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:52.959850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:52.978336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:52.981479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e372b81d-d92f-4c25-82d7-00f2415458c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:53.013820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:53.014822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:53.016521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:53.016521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:53.016521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:53.016521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:53.016521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:53.016819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22207c-b785-4651-84db-3805cf852025 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.017034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22207c-b785-4651-84db-3805cf852025 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.017278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22207c-b785-4651-84db-3805cf852025 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.022777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 339/692] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:53.024173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22207c-b785-4651-84db-3805cf852025 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.087589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.092195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90768b4-c8e2-48ee-a543-a2ed9cc76ec2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.140612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.141588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.142061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.142545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.143889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.143889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.145914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3adb3-275f-4952-9c91-1686e1167061 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.151043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22207c-b785-4651-84db-3805cf852025 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.160514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 354/693] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.161651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.161916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.162204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.183801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:53.185794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:53.199132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.199866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.200275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.200556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.201340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.201650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.203331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.217108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 340/694] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.218226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.265984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.281945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 355/695] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:53.283256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.331130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91f6ac-730c-4227-babd-64d4c147a610 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.377585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.378329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.379176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.379470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.386310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.389177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.391127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.404750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74efbf5-c971-4630-b743-8a4b9d65cfe1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:53.410422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:53.412183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 341/696] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:53.414164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.456659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.457512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.458332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.458617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.468759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.470568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:53.491068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:53.492278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:02:53.493697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:53.496458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 356/697] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:53.496458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.538675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.542182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9b738-b6ca-48dd-ad86-808ceca30c99 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.593233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.594084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.594374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.594656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.611012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.612586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.632061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e683b7-411e-4de4-aef5-aa57b203157a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:53.633670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:53.634514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:53.635464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:53.635464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:53.635464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:53.635464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:53.637657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 342/698] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:53.638695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.669112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.669794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.670082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.670361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.671297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.671613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.673186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.686720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 357/699] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.687724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.697540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.743892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959eeda0-dd77-473b-bb94-6880ce6aec56 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.778567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.779650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.780128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.780594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.781620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.782140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.783813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.797091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f6f89-d8e4-44fd-bc00-f24b6fad1b0c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.803033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 343/700] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.804119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.816482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.834973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.835780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.836072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.836353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.837175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.837486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.838883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.851893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 358/701] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.852704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.907735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a79e5-ebf4-410e-a57a-321fe8a42c4a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:53.921993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:53.942628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:53.943514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:53.943895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:53.944296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:53.945253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=0857fc0c4021456b88288ba10921ca6d) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:53.945617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:53.947189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:53.953665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 344/702] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/role_assignments?user.id=0857fc0c4021456b88288ba10921ca6d&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:53.954614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:53.963041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eceb5d-a405-4a40-8c5b-aa2f1c254045 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:54.003426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:54.004466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.005014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.005478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.059558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:54.075129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633c8b5-e02a-46b3-b1f2-1bcb0a78c9f9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:54.119930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:54.120756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.121131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.121470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.122254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:54.123833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:54.130653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:54.132772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:54.163913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 345/703] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:02:53 2022] POST /identity/v3/users => generated 254 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:54.211863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:54.255359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 359/704] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:54.260451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:54.372132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a23b-4934-4417-b2ea-f3f7eea8582e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:54.403477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:54.410940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:54.412172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.412771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.413387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.424580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:54.426001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1de44bd7-36be-4a3e-9d76-7b08b26ed18e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:54.444105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:54.445043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:54.446105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:54.446105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:54.446105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:54.446105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:54.446105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 346/705] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:54.450837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:54.620974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6e96e-e2d0-48ce-a8ff-463534b8ddd5 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:54.634477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a90969-f8a4-4333-a698-117fdc33b0f8 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.634813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a90969-f8a4-4333-a698-117fdc33b0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.635039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a90969-f8a4-4333-a698-117fdc33b0f8 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.636728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 347/706] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:54.642006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.642296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.642578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.669368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:54.670468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.670972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.672032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.672993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:54.673513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:54.675253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:54.688702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 360/707] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:54.691125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:54.692765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:54.693845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:54.727051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:54.817023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 348/708] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:54.866000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:54.866803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac856-464f-4faa-a097-44980f8c675a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:54.918266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:54.920367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:54.920367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:54.920367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:54.934322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:54.936060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:54.958883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:54.960041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:54.960986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:54.963000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 361/709] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:54.968792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.052857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e6e11-f6ec-4464-a2eb-b484bef52b55 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.077440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:55.110049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:55.123607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.124693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.125265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.125630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.126439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.128020 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.139236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72128c-c781-4d11-ad08-7213f87e0064 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:55.156101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:55.157367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:02:55.158459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:55.165052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 349/710] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 30 04:02:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:55.170394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.211203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.213095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.213689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.215160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.216141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:55.216798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.219183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.244092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 362/711] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:55.252151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.330342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a41060-97bd-4e91-a40e-96450fc36f95 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.364336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5cf63-c78a-4bfa-979a-dcc8d0525d02 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.369317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.370221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.370702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.371790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.372685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:55.373610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.375614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa50277-bf90-4e76-88e2-93ef8dc9418d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.381996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 350/712] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:55.386599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51686b-ff9f-4ec7-b826-c8cda2494381 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.386599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51686b-ff9f-4ec7-b826-c8cda2494381 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.386599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51686b-ff9f-4ec7-b826-c8cda2494381 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.390204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 351/713] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:55.391635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51686b-ff9f-4ec7-b826-c8cda2494381 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.409289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.409985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.410420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.410816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.411691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:55.412006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.413672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d5bcb-6725-497e-af63-70d6ba7e9298 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.423144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 363/714] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:55.423144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.423144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.423144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.444075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:55.445604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:55.450162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:55.483777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:55.563652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 364/715] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:55.594469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51686b-ff9f-4ec7-b826-c8cda2494381 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.612741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.642641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.643829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.644517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.644987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.645901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:55.646436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.648184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.655984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 352/716] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:55.724085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95f2c5-63eb-4698-8818-b3b3e0448245 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.759679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.760423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.760708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.760982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] PATH_INFO: `/v3/services/network` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.761660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.763119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:55.777930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:55.779039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:55.779933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 365/717] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:55.787293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.915603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9f2bb6-4fe8-4699-afa6-44d3d58af08f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:55.918329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:55.952416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:55.953436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:55.953912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:55.954375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:55.955323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] RBAC: Adding query filter params (name=network) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:55.955822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:55.957644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128dc2-1d75-4df3-9a7d-71f2fbdc2336 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:55.962516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 353/718] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:55.968262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:55.972612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:56.109958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53adb0-2b42-4124-bda2-7abbb8275c92 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:56.145109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:56.145864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.146075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.146351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.147081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] RBAC: Adding query filter params (type=network) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:56.147409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:56.148816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9dc5a9-a4ee-46f6-8590-e9c3f8dd3c36 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:56.155406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 366/719] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:02:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:56.552725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879e2e-5bf6-4274-8185-d086ac836ed7 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.553291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879e2e-5bf6-4274-8185-d086ac836ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.553291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879e2e-5bf6-4274-8185-d086ac836ed7 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.554924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 354/720] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:56.560466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7cb95-f630-4893-b2ba-767c7497067c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.560760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7cb95-f630-4893-b2ba-767c7497067c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.561040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7cb95-f630-4893-b2ba-767c7497067c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.579760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-17a7cb95-f630-4893-b2ba-767c7497067c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:56.581285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-17a7cb95-f630-4893-b2ba-767c7497067c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:56.673661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:56.681311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 367/721] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:56.717219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15879e2e-5bf6-4274-8185-d086ac836ed7 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:56.728779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe7b00-41fe-4311-b8f8-3016f90d5b4e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.729064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe7b00-41fe-4311-b8f8-3016f90d5b4e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.729355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe7b00-41fe-4311-b8f8-3016f90d5b4e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.731611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 368/722] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:56.737431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.737723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.738222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.757489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:56.758694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:56.859418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 369/723] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:56.907248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15879e2e-5bf6-4274-8185-d086ac836ed7 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:56.939550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:56.967497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:56.968321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:56.968619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:56.969140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:56.988822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:56.996883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:56.997819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:56.998655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 355/724] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:57.004676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.045554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.132860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d7c8-96d4-4d60-95ad-072553c62647 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.153180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d777df0c-5096-448a-8fe3-435d746e1851 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.170646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.171957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.172356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.172727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.179114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:57.179114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:57.179114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:57.186355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 356/725] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:02:57.193000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.221184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.236304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.237119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.237397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.237669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.258395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:57.270720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:57.271825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:57.272675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 370/726] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:02:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:57.279349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.362922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.380931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21a179-df70-460f-bda7-58c958a9a2bb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.443932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.445101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.445585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.446056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.453282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187b9ea-a207-402c-9059-92cd72f94054 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.476304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:57.477827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:57.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:57.504082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:57.504974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:57.504974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:57.504974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:57.504974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:57.505982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 357/727] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:57.515050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.515762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.516042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.516315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.517128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:57.517433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:57.518861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:57.521496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.537284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 371/728] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:57.542601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.632003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.717269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4087-c4c6-4795-a21e-04b2735a3ca5 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.721114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ed9df-bfd1-4342-947d-8c23dbf2e12a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.779148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.779832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.780112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.780382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.781110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:57.782312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:57.784295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:57.788903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:57.789731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:57.790008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:57.790281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:57.809408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:57.811016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:57.816393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 358/729] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:57.826677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.829712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:57.852598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:57.854662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:57.856871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:57.856871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:57.856871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:57.856871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:57.856871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 372/730] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:02:57.862963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:57.974932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74719b53-4f6c-4b17-9e09-118fd7022872 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:57.976460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:57.984089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0ec6d-5d9c-4f95-92c7-f12b693e1b2a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.009992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.010665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.010946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.011266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.011990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:58.012294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.013677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56129c4-5d41-4045-99d2-d6c1b5b65fbf None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.023451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.024268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.024554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.024835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.025608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 373/731] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:58.026969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb228b-8326-46b4-8613-4ab7731feebc None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.027279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb228b-8326-46b4-8613-4ab7731feebc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.027559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb228b-8326-46b4-8613-4ab7731feebc None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.033167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 374/732] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:58.033343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.034430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.034807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.035144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.035295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:58.050887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:58.053402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:02:58.055716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:58.056959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:58.056959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:58.057239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 359/733] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:58.057239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.120658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.131281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 375/734] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 04:02:58.132235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.163398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cd294-c87d-4c5b-8ab9-948b393d30a6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.200423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.201251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.201557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.201804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.209404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.211029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.224815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-43b34ea4-a254-465a-806e-1c4db85d1113 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:02:58.226339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:58.227738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 360/735] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:58.228700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c267f-7241-47ba-9c11-122f3e891e5f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.229352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c267f-7241-47ba-9c11-122f3e891e5f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.229352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7c267f-7241-47ba-9c11-122f3e891e5f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.234870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 361/736] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:58.234870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c267f-7241-47ba-9c11-122f3e891e5f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.239490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1dac4-5096-4aff-870b-b51bfc0aadd3 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.276309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.276996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.277287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.277566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.278416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:58.278742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.280197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.298215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 376/737] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:58.299049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.345347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7c267f-7241-47ba-9c11-122f3e891e5f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.370132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.386210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.387087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.387403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.387656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.398498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1b43c2ad-98b8-4cda-a998-2272e5d01e7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:02:58.406291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:58.407320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:58.408180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 362/738] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:58.408883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.409536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.409536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.417913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3208fdc-642f-4c63-bc91-29e296a77580 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.428377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:58.429587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:58.457191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.457887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.458293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.458579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.459463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:58.459780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.461457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.484783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 377/739] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:58.485473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.497437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.500670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 363/740] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:58.501702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.607949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25b917-35e8-4b43-ad17-cd36534d61fc None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.623037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481e942-b4c6-4280-836d-1c6d7bf1a922 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.636668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.646877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.647523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.647806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.648080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.662209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.662893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.663213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.663518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.664261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:58.664564 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.665969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.675129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 364/741] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:58.680588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.719958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.721436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.731743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.787985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cf8f6-3c4c-4087-82c0-39ef3b0dfe56 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.823746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.824403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.824688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.824964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.830082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 378/742] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:58.830900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.866669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.892091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.893566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be71c-7a09-429d-95ac-8735e541448c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.929927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d80ea2-097a-4a7f-9d8d-2fcdbd75d904 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:58.938947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:58.964208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:58.964905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.965187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.965459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.966181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:58.967577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:58.982260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 379/743] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:02:58 2022] POST /identity/v3/services => generated 251 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:58.983111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:58.991729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 365/744] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 04:02:58.993138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894b2e7-aedc-4aa7-a795-86c3062499a5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:58.993431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894b2e7-aedc-4aa7-a795-86c3062499a5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:58.993711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8894b2e7-aedc-4aa7-a795-86c3062499a5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:58.995334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 366/745] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:02:59.000102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894b2e7-aedc-4aa7-a795-86c3062499a5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.089009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.127908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8894b2e7-aedc-4aa7-a795-86c3062499a5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.132101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540fded-56b1-4b8c-8a4c-33781f622850 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.169728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.170539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.170746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.170859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.171080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.171840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.172230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.172620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.182735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.184280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.184864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.186329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-4dffeaec-9413-456a-8917-c1491b5d66e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:59.202590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:59.203728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:59.204803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:59.204803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:59.204803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.204803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:59.205166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:59.206037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 367/746] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:59.206987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.208701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 380/747] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:59.210070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.210490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.210892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.228041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:02:59.229237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:02:59.241299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.301556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 381/748] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:02:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 04:02:59.302763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.331141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f40b5-35c9-400d-bd01-1db93746951f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.360034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.377360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.378193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.378651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.379460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.391808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.393341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:02:59.411391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:59.412497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:59.413431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:02:59.413431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:02:59.413431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:02:59.413431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:02:59.413431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 368/749] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:59.413825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.421163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e93d8b-0100-41a2-a119-6d3bf6a257f5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.459166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.459880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.460169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.461181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:59.461493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.462915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.475088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 382/750] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:59.475940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.485638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.536846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3ff5f-3d8d-4611-9c04-f16fbcb885ee None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.582295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.582979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.583354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.583642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.584395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:59.584710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.586094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b2842-8ada-4c69-8e54-58e79933836b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.586197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.601028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 369/751] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:59.601842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.625067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.625665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.625950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.626227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.626955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.628458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.641043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.642893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 383/752] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:02:59 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:02:59.644038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.741588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f71f98-ac71-4bbe-b5a4-e4a9da52094b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.743520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.795464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.796948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.797421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.797880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.798874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:02:59.799547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.800152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d2138-f997-4720-a6f1-02c1053e108d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.802057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.818072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 370/753] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:02:59.819416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.846200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:02:59.847485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:02:59.848113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:02:59.848621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:02:59.857291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:02:59.859483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:02:59.874667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:02:59.875834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:02:59.876911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 384/754] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:02:59.877474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:02:59.935442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:02:59.949544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac0f26-70e2-4a6b-8dea-3103f78066a4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:02:59.993870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e780-8a48-458b-80c2-17a17d5f3928 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.001390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.002877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.003381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.004437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.015171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.017345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.020049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.034457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.036145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.036729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.037299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:00.038798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:00.039913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:00.041093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:00.044136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 371/755] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:00.045112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.052418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.054169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:00.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:00.071258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:00.072176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 385/756] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:02:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:00.073418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.165517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587a601-11ed-4354-9218-6c8e7419a891 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.181379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7785e26-6678-4dc0-8f46-3ba772a954c7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.203290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.203959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.204243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.204519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.205306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.205715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.207800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.217617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.218393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.218751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.219127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.220055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.220435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.222029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.225492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 372/757] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.226415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.243427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.245595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 386/758] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.246502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.293716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.342495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30cc36-f7f2-47cb-a100-c0b6785b2a11 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.375125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db826e1e-e378-45d0-8bc3-260fdb4fec15 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.378478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.379243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.379624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.379974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.380916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.381294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.382868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.400873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 373/759] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.400873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.425154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.427723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.428154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.428569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.430196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=0857fc0c4021456b88288ba10921ca6d) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.430667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.432746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.447741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 387/760] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/role_assignments?user.id=0857fc0c4021456b88288ba10921ca6d&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.450681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.509978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.575752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c205d4-2304-496e-94f7-25f3a0a2abb0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.654551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.654551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.654551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.654930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.655001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf214472-8a7a-49ed-bca1-5093f8626133 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.658381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.676651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.678367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:00.697695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:00.698662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:00.699959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:00.699959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:00.699959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:00.699959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:00.699959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 374/761] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:00.710234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:00.715436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.716353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.716825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.717319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.718359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=8079221b336646d3af1dc7aba9719ad0) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.718875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.721516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.728845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 388/762] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/role_assignments?user.id=8079221b336646d3af1dc7aba9719ad0&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.844347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ca9b1-e807-4b5c-b65b-086d0bc76cb8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:00.870420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:00.886756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:00.887480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:00.887762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:00.888041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:00.888784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:00.889093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:00.890491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68606c71-63a3-457b-83f5-081886ddd1dd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:00.911498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 375/763] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:00.917600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.045453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd066e85-032b-4bc9-b0e9-d6a84f448da1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.081214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.082090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.082377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.082655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.089900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.091397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.102160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db624fa-714c-4c28-9b3c-bc9717b84b71 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.102443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db624fa-714c-4c28-9b3c-bc9717b84b71 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.102727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db624fa-714c-4c28-9b3c-bc9717b84b71 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.104390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 376/764] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:01.109134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:01.110017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:01.110910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:01.111382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.111382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.111382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.112497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 389/765] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:01.118809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.129636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:01.131186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:01.152501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:01.228546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 377/766] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:01.239287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0b2fa-f55c-4c4e-ad25-edd2121f7f0e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.290670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.291431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.291657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.291917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.292725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:01.293036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.294498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.309715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 390/767] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:01.310377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.326664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.420198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08473bf3-b4f3-475c-9860-47cd1a6ad22f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.424122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:01.437821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5782-be25-46ee-aeb3-fb651b6314b7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.451457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:01.474816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.476140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.476671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.477188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.478057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.479784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:01.493689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:01.494711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:01.495568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:01.498573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 378/768] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 30 04:03:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:01.505895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.515749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.517103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.518778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.519331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.521847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=67b4385cb5ed42d1b9488d409abd61b1, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.526671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.558295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 391/769] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:01 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:03:01.649831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a0deb-910b-4691-a2ed-84e7e55715bd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.686632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.687784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.688318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.688802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.689718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:01.690221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.691917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.697965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 379/770] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:03:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:01.704358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.838756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b45a75-e980-4700-8c08-c889324c716e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:01.878771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:01.879977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:01.880485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:01.880958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:01.881888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:01.882409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:01.884098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa85f7-e7e6-4f95-beb4-d66de0a3c83e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:01.891777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 392/771] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:03:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:01.900603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:01.955456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:02.016165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b561d6-abbb-449c-a986-f0d1b0f70f87 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.016520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b561d6-abbb-449c-a986-f0d1b0f70f87 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.016765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b561d6-abbb-449c-a986-f0d1b0f70f87 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.018624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 393/772] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:02.025043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.025339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.025621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.040315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b24ad7-0a57-444e-8f3e-8187de72eea3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.049515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:02.050844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:02.069197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:02.109670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.110267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.110551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.110830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.111651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] RBAC: Adding query filter params (interface=public, service_id=b61712201ee84d0fbc82aaf031e25771, region_id=RegionOne) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:02.111964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.113611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.129936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 380/773] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:01 2022] GET /identity/v3/endpoints?service_id=b61712201ee84d0fbc82aaf031e25771&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:02.135785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:02.192486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 394/774] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:02.216094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:02.297894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb18a7-943c-4489-8e87-18622c4f12c2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.301379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:02.340089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.341176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.341653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.342117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.343034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.344642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.350902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 381/775] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/services => generated 772 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:02.397232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a1ec7-3a8d-4849-9709-2e4a771147c7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.436086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.437293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.437826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.438290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] PATH_INFO: `/v3/services/network` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.439209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.440878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:02.454090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:02.455213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:02.456052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:02.463217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 395/776] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:02.465390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:02.577085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82dd0c-7a7b-4042-8c35-bf21c97dfa4a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.613884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.614574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.614849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.615146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.615865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] RBAC: Adding query filter params (name=network) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:02.616173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.617556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.627278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 382/777] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:02.628956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:02.638956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:02.740630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390607fb-f945-49e3-bb59-4cd4dca5af4f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.777402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.778326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.778772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.779223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.780077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] RBAC: Adding query filter params (type=network) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:02.780520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.782102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8d8573-0add-443e-8963-3d5c471dbc5b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.792619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 396/778] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:02.794224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:02.828128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:02.902631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6679788-de0d-41c5-af32-1ecfc0e8bb41 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:02.916528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d907-00cd-4b59-b9c6-b2e99c9e3669 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.917145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d907-00cd-4b59-b9c6-b2e99c9e3669 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.917915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d907-00cd-4b59-b9c6-b2e99c9e3669 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.923636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 397/779] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:02.927009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.927559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.928512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.955202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:02.956474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:02.962043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:02.962721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:02.963009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:02.963309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:02.964109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] RBAC: Adding query filter params (interface=public, service_id=dd8ce1fb388047668ef8805035bf040d, region_id=RegionOne) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:02.964424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:02.965802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:02.977855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 383/780] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/endpoints?service_id=dd8ce1fb388047668ef8805035bf040d&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:02.984227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.108477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:03.116125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:03.117989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be329694-bd91-4b63-86e4-7cfd8a19030b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.135132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 398/781] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:03.180450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.181153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.181441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.181717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.182410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:03.183821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29463499-e28a-4505-9f03-5184b69a3825 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:03.184177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.189828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 384/782] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:02 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:03.320866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93440d-c661-4264-b1a5-a3b5a2b7fdd9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.357283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.358105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.358288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794848-051d-42c9-ba9f-f5c1d8a15af4 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.358386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.358634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794848-051d-42c9-ba9f-f5c1d8a15af4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.358731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.358896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a794848-051d-42c9-ba9f-f5c1d8a15af4 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.360531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 385/783] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:03.366336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.366626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.366910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.381013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:03.385626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:03.386841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:03.388812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:03.389749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:03.392583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 399/784] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:03.396970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.473528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 386/785] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:03.485036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:03.496150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.529514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996baf10-9c28-4832-86ed-3e1f1398c1fb None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.585092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.585807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.586095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.586373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.587168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:03.587481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:03.588894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:03.604846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 400/786] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:03.612249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:03.614802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.619730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7383b-71f3-4b2e-a829-908e86ce833f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.662447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.663346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.663685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.663969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.677379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:03.691126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:03.692326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:03.693178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 387/787] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:03.694384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.779716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f41f90-3c31-4f5f-b59c-9ae2d8bdec02 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.807129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:03.817000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.817822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.818105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.818383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.829463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:03.830919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:03.848299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:03.849246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:03.850146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:03.850146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:03.850146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:03.850146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:03.850146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 401/788] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:03.857855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:03.910845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be669-c86f-4ce5-a19f-3137f3e3a4f3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:03.985859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:03.986569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:03.986851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:03.987163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:03.988188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:03.988576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:03.990770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.005609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 388/789] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:04.010736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.020142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ed45e-43a7-47eb-8345-dc340b59602a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.044415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.073611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.074707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.075267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.075758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.076694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:04.077252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.078901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.092606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 402/790] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:04.100168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.137106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.186284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67d0042-581d-4e4b-92bf-5b53e5f3819d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.213532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290a73e-5c5b-4ecf-bc46-75cb2b901d02 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.227478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.228900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.229406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.230320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.244228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.245823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.254200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.255409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.255838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.256269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.264502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-eac6826b-0869-4017-82c1-9328c035a983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:04.264682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:04.265764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:04.266884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:04.266884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:04.266884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:04.266884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:04.268541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 389/791] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:04.268983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.270590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715326a-2190-4bd7-b08e-043f85df77e2 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.270980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715326a-2190-4bd7-b08e-043f85df77e2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.271437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715326a-2190-4bd7-b08e-043f85df77e2 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.273114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 390/792] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:04.273916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715326a-2190-4bd7-b08e-043f85df77e2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-fc7a5315-3dcf-4edd-ba3c-fbba2f99ed99 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:04.286149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:04.287356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:03:04.288523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:04.290587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 403/793] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:04.291965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.292378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.292821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.312337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:04.313802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:04.357362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.398129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 404/794] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:04.399433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.406118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715326a-2190-4bd7-b08e-043f85df77e2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.432524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.449071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.450294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.450703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.451083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.451946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:04.452356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.454750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.474718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 391/795] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:04.476186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.518749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486e48b-e1a3-4019-9740-7c18d850e398 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.567168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.567168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.567168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.567168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.568349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:04.568790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.570406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.595199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 405/796] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:04.596293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.610310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705fe50-ea87-45fc-9394-655919cf6583 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.636194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.649736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.650750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.651208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.651626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.663069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:04.671244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:04.672237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:04.673097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:04.676337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 392/797] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:04.677442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.707857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.732292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2444-d7e8-4328-8d74-0fb1197f49cc None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.771344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.772292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.772574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.772852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.780804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.782379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.797345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c80fb3-a700-41af-a5eb-6ce108e0c820 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:04.798643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:04.799613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:04.800758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 406/798] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:04.802189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.839005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.839947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.840330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.840614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.852022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.853581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:04.872329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:04.874479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 393/799] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:04.876725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.914607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bb1c-dc84-41c7-bb36-425805f5ed25 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:04.921483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:04.952161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:04.952945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:04.953235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:04.953512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:04.954364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:04.954680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:04.956109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:04.970168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 407/800] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:04.971084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:04.975063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.006846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f8f3a-c992-4fcb-b8b4-cd3b8d11a68f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.048672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.049360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.049643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.049919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.051359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:05.051673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.053933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.075614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 394/801] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:05.076846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.086152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b110a19-52df-4ef5-8079-aaad6d1bd9df None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.124268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.124952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.125243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.125526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.126257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:05.126567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.128004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.143267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 408/802] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:05.143267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.180207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.188373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d628de-923d-41be-adab-6f88e22fccd8 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.210534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.230327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.231035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.231340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.231647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.241981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.243486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.250983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930184b-8064-438a-885e-58c0d0dfeb25 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-0d0f104d-eac3-44a0-8fb5-60475b65bd14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:05.262872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:05.264085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:05.265000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:05.265000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:05.265000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:05.265000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:05.265000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 395/803] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:05.265648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da149976-4268-4a99-9ec2-3e232b937aeb None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.265716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da149976-4268-4a99-9ec2-3e232b937aeb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.266091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da149976-4268-4a99-9ec2-3e232b937aeb None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.271145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 396/804] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:05.271145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da149976-4268-4a99-9ec2-3e232b937aeb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.296466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.297506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.297939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.298393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.309576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.311507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:05.328985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:05.329943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:05.330826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:05.330826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:05.330826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:05.330826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:05.330826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 409/805] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:05.331143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.395660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da149976-4268-4a99-9ec2-3e232b937aeb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.431020 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.431863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.432146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.432418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.438966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.440481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.444814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1093b-f51b-410b-a512-a9bd39277a8e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1c4c127d-90c4-47dd-8c0b-1a297c933d6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:05.454702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:05.455720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:05.456706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 397/806] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:05.458241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.458530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.458810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.468029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.471257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.477270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:05.478479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:05.487525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.488792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.489072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.489345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.490077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:05.490379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.491781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.507625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 410/807] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:05.507625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.551930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 398/808] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:05.552689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.628450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c68609-5b8e-4eb5-b6b3-8d8d4651906b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.657336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd9721e-f779-4136-b94b-c617f19c3880 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.663388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.664065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.664348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.664625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.665355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:05.665657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.667026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.685797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 411/809] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:05.685797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.698054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.698951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.699231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.699569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.700435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:05.700809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.702338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.705050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.715693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 399/810] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:05.716517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.722233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.785965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891acc-70e9-4c72-bc58-ae260d113e94 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.822309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.823136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.823424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.823699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.823830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe61758-4bbb-4e07-8739-c07355263ba4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.831038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.832942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:05.847295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:05.853455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 412/811] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:05.854602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.864615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.865446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.865740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.866018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.866688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:05.868100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-2ed6bf5a-c3f1-4a14-a6be-4207a37e1dff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:05.881003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:05.882075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:05.883083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 400/812] 10.222.0.13 () {56 vars in 1129 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:05.884084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944805f1-33b3-41e6-956c-cffbd1c4179f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.884084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944805f1-33b3-41e6-956c-cffbd1c4179f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.884084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944805f1-33b3-41e6-956c-cffbd1c4179f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.885257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 401/813] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:05.885936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944805f1-33b3-41e6-956c-cffbd1c4179f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:05.946942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.947608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:05.953923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0284eac-9ec0-4894-a609-f32c9ba84eae None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.990275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:05.991090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:05.991361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944805f1-33b3-41e6-956c-cffbd1c4179f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:05.991466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:05.991668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:05.998609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.000137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:06.014364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:06.015445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:06.016504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:06.019245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 413/814] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:06.020274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.031927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.032832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.033119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.033399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/0857fc0c4021456b88288ba10921ca6d/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.035683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=0857fc0c4021456b88288ba10921ca6d, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.037203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.052128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 402/815] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:05 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/0857fc0c4021456b88288ba10921ca6d/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:03:06.053482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.142852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1c273-fde8-4709-81eb-6b0bffd1e76a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.177451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0de227-6e13-4ad6-9b72-8c05652a3a7d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.182162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.182862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.183177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.183454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.184313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:06.184681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.186242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f350f5-5d30-464e-90ae-9eee09107cfa None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.198866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.212000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 414/816] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:06.214521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.214521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.214738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.221805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.222899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.223309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.223866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.224347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:06.225128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.226608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.231666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 403/817] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:06.232220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:06.233061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.233421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:06.243649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.311210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 415/818] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:06.311210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.353273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb634632-9edc-49e4-a34a-4aebe36efae1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.389108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.389784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.390066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.390340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.391179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:06.391489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.392922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.410615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 404/819] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:06.410615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.421233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7024b9-0ea8-4321-bbcf-c69029a995b5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.462026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.462706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.462845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.462992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.463303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.480753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.526175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cfefe-a66e-4f91-8861-e585550b9995 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.535409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.536877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.562067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.562665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.562947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.563253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.563967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:06.564269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.565657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.571184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 405/820] 10.222.0.13 () {56 vars in 1124 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:06.571997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.648878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 416/821] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:06.650402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.689087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5def5-2acb-4b30-a155-9e580ecb5ad1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.699578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.700912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:06.729782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.730930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.731425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.731894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] PATH_INFO: `/v3/services/network` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.733233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.734943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:06.756024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:06.757099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 04:03:06.763389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:06.764310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b29ae7-4cd5-4832-9df4-445ae30fc647 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.764406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 406/822] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:06.764406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.806050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.807507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.807791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.808067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/8079221b336646d3af1dc7aba9719ad0/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.872563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e95f9-7fca-4432-b8d9-55dd073ae225 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:06.911321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:06.912013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:06.912294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:06.912575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:06.913337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.914741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.947128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 407/823] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:06.947128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:06.949466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=8079221b336646d3af1dc7aba9719ad0, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:06.950992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:06.967035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 417/824] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:06 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/8079221b336646d3af1dc7aba9719ad0/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:03:06.967977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.021071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.138101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.149364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c127d17-240d-47d2-82f8-3d5c0c2f2775 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.200833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.201211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.201492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.201764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.203083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] RBAC: Adding query filter params (name=network) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:07.203650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.204781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.213551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 418/825] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:07.216327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb5789-e38d-4616-9158-4ff4b736c8b2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.219319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.262757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.263988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.264555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.265060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.280540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.283505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:07.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:07.308406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:07.309309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:07.309309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:07.309309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:07.309309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:07.312062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 408/826] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:07.316774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.341486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28f18-fdbd-425b-b9db-44de7ffd8b5c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.394926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.395918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.398034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.398466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.399355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] RBAC: Adding query filter params (type=network) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:07.399809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.402273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.411206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 419/827] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:07.416887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.433953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.446082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.493050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77914f-37c1-4a9f-9321-8f4f20869864 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.525104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dae3c1-76ed-4565-b049-dad774a43c15 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.561267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.561483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.562087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.562437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.562781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.562884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.563430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.563682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.564098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.565039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:07.565140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.565686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.572389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.592298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 409/828] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:07.598746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.617008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 420/829] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:07.748261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941100f7-224a-490b-b7cb-9027be4a8ff1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:07.807985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:07.808852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:07.809180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:07.809466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:07.813384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.818551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:07.820129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:07.836383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:07.837367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:07.838352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 410/830] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:07.845474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:07.968100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:07.997969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa91272-4da6-4f29-9ca8-ccd8b0aea9b1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:08.035432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:08.036030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.036320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.036599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.037396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:08.037707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:08.039172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1633119-c6b8-4525-b4b6-eec1bdf83e9f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:08.052864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 421/831] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:08.058712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:08.162496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cfc8-d7e0-4910-8f3e-61679d11ed58 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:08.197642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:08.198315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.198600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.198880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.199753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:08.200182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:08.201633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d1a8c-45cc-4d9f-9703-81206feb805e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:08.208602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 411/832] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:08 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:08.221204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1add9-3c5b-4775-813c-91a6e9ef5b7e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.221524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1add9-3c5b-4775-813c-91a6e9ef5b7e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.221785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1add9-3c5b-4775-813c-91a6e9ef5b7e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.224063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 422/833] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:08.230090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.230399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.230679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.249859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:08.251041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:08.319934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:08.339234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 412/834] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:08.356986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1add9-3c5b-4775-813c-91a6e9ef5b7e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:08.472852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1add9-3c5b-4775-813c-91a6e9ef5b7e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:08.509494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:08.510306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.510588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.510864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.512850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:08.523602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:08.531386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:08.532318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:08.533191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:08.536118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 423/835] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:08.539170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:08.657212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3342d18-373e-422b-b76a-9194af18ec44 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:08.693066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:08.693750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.694036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.694310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.695055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:08.695389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:08.696828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:08.710742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 413/836] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:08.712201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:08.814418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f7415-7881-46dc-b857-f3832932cb53 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:08.849235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:08.850106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:08.850391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:08.850669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:08.861570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:08.863753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:08.881306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:08.883415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:08.885581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:08.885581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:08.885581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:08.885581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:08.889624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 424/837] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:08.889814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:08.945951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:08.993841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03134c4-fdf5-44a8-9edc-1548d7286128 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.029001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.029743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.030026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.030301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.031085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:09.031421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.032876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966844a-b247-41cc-b2cd-ff88a77ba9ee None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.050505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 414/838] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:09.052318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.103833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.161468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8adcd-252f-4020-82dd-7aa62d748745 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.197669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.198406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.198685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.198965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.202665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82acd9-8c44-46af-99d5-dc964ecbce9e None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.203128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82acd9-8c44-46af-99d5-dc964ecbce9e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.203335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82acd9-8c44-46af-99d5-dc964ecbce9e None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.204903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 415/839] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:09.207427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.208884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.213560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.213560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.213560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:09.227254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:09.228280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:09.229220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 425/840] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:09.236418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.348160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378df3d6-b05e-481f-b610-66ed63d7dcc6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.357479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.360920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:09.363269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:09.389391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.390171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.390457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.390731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.391602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:09.392009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.393597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.414594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 426/841] 10.222.0.13 () {56 vars in 1135 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:09.420030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.437725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 416/842] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:09.461434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.504656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.529914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceefdab-2c00-4523-8175-6c8dc00ae1df None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.570433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.570433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.570433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.570433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.574781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda12b9-b97c-4ed1-938e-ef5df2e48994 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.606853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.609569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.610379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.610657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.610928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.621868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ea7d9271-b1ce-43ab-b30e-e581aade272c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:09.629682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:09.630827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:09.631737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 417/843] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:09.632488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762578b-4870-4626-8904-d37453a75d6f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.632784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762578b-4870-4626-8904-d37453a75d6f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.633109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762578b-4870-4626-8904-d37453a75d6f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.638568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 418/844] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:09.639349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762578b-4870-4626-8904-d37453a75d6f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.654442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.655915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d77d86a-0949-404f-a80f-c3e47179a85e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.746313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762578b-4870-4626-8904-d37453a75d6f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.747911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.763698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 427/845] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:09.765353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.765792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.766213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.780770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.781444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.781727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.782004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.782753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:09.783059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.784621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.785346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:09.787348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:09.794430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 419/846] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:09.795452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.857194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:09.866534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 428/847] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:09.867393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.905200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7004d9f3-30f8-41a3-b95a-1d647c94a623 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:09.941666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:09.941990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:09.942255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:09.953282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:09.955306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:09.974355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:09.975377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:09.976314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:09.976314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:09.976314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:09.976314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:09.976314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 420/848] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:09.976611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:09.996416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39a8bf-1a3c-4abe-8832-e84b2f52aac9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:09.999269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.035238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.036498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.036862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.037207 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.047922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.049397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:10.068775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:10.069772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 429/849] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:10.070669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.100010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b83edc-58ce-4506-93c3-6edfa0256d0a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.134861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.142888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.143939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.144222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.144499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.145174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.146561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:10.159715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:10.161210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:10.162042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 421/850] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:10.162940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.200892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac19c3-a15d-4452-9608-142db082cd56 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.240167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.240850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.241135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.241410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.242136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.242443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.243858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.256110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 430/851] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.256877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.275804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.299556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe6967-53a0-4a35-aaac-bd0260fdaaa3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.343892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.344559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.344845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.345125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.345862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.346171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.347647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.361448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 422/852] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.363182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.386926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a300053-b69e-4f44-95bd-7041f430e1c6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.394998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.427432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.428117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.428402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.428678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.429513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.429956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.431343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.437045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 431/853] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.437859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.475482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679b6b7-34bd-4b7c-a4e8-e5e61fd80ae6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.512313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.513027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.513311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.513589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.522860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.524695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.527449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:10.540773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:10.541959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 423/854] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:10.542912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.547976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc429aac-4ded-402c-9471-212a5c33c880 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.591797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.592635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.592913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.593204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.601295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.602788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-a5029d67-7596-4405-9a57-421f2cd655d0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:10.617903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:10.618998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:10.619980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 432/855] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:10.620562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8baf3a-63e6-4402-a38a-52ee236106d3 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.620628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8baf3a-63e6-4402-a38a-52ee236106d3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.620978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8baf3a-63e6-4402-a38a-52ee236106d3 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.622103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 433/856] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:10.622788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8baf3a-63e6-4402-a38a-52ee236106d3 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.656405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.658262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9047b5-af0c-4b61-bd90-8b78a75ebd5e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.695016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.695734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.696028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.696313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.697067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.697382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.698781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.704279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 424/857] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.705086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.750945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8baf3a-63e6-4402-a38a-52ee236106d3 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.783472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:10.793529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.795346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.796003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.796747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.797778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.798200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.800634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d4326-fac4-48d8-b942-8479cda39104 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.815290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 434/858] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.817069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.818656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.818656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.838184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:10.839595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:10.840924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c52549-c75b-4294-a8f2-78e4c2428aaf None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:10.890627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:10.891377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:10.891668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:10.892239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:10.892799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:10.893120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:10.894706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:10.929380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 425/859] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:10.929380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.937386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 435/860] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:10.944314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:10.973539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.032992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468390e-4e9e-4e83-8d55-1d17005ea16a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.059372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.060181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57caa7-c335-46d1-ab69-fd3f6b439861 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.070884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.071718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.072103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.072490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.073412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=0857fc0c4021456b88288ba10921ca6d) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:11.073829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.075537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.082180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 426/861] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/role_assignments?user.id=0857fc0c4021456b88288ba10921ca6d&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:11.083144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.099257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.100226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.100655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.101076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.208943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.210652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.225519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a62e5-4585-43df-b1d6-2cbbfacc1861 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.242924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.268100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.269063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.269347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.269618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.270460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.271915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:11.285866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:11.287322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:11.288163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 427/862] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:11.292991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.347849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 436/863] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:11.353019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.359249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.420354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437785da-c79f-40bf-b5eb-5cc61d6f6055 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.458045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.459284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.459822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.460318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.461235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:11.461742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.463526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.468909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 428/864] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:11.474195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.481936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d41ea-38a0-486a-b2bc-601db9fe3443 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.508825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.531410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.532229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.532515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.533167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.546284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.548377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:11.574364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:11.575512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:11.576406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:11.576406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:11.576406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:11.576406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:11.581395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 437/865] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:11.581395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.614286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdaa1d8-457c-4594-a664-fa506ea3164a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.629010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:11.652477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.653683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.654676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.655160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.656292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:11.656747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.659463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.665852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 429/866] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:11.697812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82915ba3-a0ad-4792-8fda-01f5c1a15a4d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.738205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.739283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.739774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.740226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.741180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:11.741636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.743235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.757632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 438/867] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:11.766794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.880462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e84e8-ba78-43cc-a654-690845b56ee0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:11.915989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:11.916802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:11.917091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:11.917375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:11.924238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:11.926226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:11.940795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:11.941727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:11.942606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 430/868] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:11.948145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:11.983772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:12.056396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e060-690f-420f-b5d3-c3485404140b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:12.093869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:12.094592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:12.094882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:12.095190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:12.096037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:12.096352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:12.097928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a22b4-6318-4561-ab10-2a022a2f2493 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:12.110553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 439/869] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:12.116710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:12.158295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:12.222581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578617-748e-4305-9fba-f934a578f8e1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:12.257120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:12.257788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:12.258068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:12.258341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:12.259204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=8079221b336646d3af1dc7aba9719ad0) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:12.259516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:12.260962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1d558-0a70-47eb-8f87-da24c9aba48c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:12.267610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 431/870] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:12 2022] GET /identity/v3/role_assignments?user.id=8079221b336646d3af1dc7aba9719ad0&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:12.769828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c882aa-6e6c-4f14-891f-60b412b8ea36 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:12.770241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c882aa-6e6c-4f14-891f-60b412b8ea36 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:12.770447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c882aa-6e6c-4f14-891f-60b412b8ea36 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:12.772094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 440/871] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:12.777528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f98da-cd27-4ce0-aa8f-3dc957703df1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:12.777996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f98da-cd27-4ce0-aa8f-3dc957703df1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:12.778073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f98da-cd27-4ce0-aa8f-3dc957703df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:12.795920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8f3f98da-cd27-4ce0-aa8f-3dc957703df1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:12.797118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8f3f98da-cd27-4ce0-aa8f-3dc957703df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:12.875286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 432/872] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:12.890840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c882aa-6e6c-4f14-891f-60b412b8ea36 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:12.988198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:13.000336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c882aa-6e6c-4f14-891f-60b412b8ea36 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:13.039531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:13.040268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.040553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.040832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.041609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:13.042987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef536c41-fa63-49e0-b02a-5deb00553dd1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:13.060306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 441/873] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:12 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:13.357819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44251-d364-4906-ad37-139379162902 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.358336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44251-d364-4906-ad37-139379162902 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.358711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e44251-d364-4906-ad37-139379162902 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.360507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 433/874] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:13.366068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.366464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.366835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.384114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:13.385315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:13.464144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 442/875] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:13.486479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e44251-d364-4906-ad37-139379162902 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:13.517702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:13.604058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e44251-d364-4906-ad37-139379162902 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:13.638927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:13.639698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.639979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.640252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.640906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:13.642280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:13.654902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:13.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:13.656789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 434/876] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:13.662329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:13.741339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:13.763734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7dd06-d484-44b4-8cef-f72a7b8498f7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:13.799116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:13.799809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.800151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.800429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.801513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:13.801820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:13.803238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6afc16-5a27-40ed-95b1-a3031097e92a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:13.808257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 443/877] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:13.813893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:13.827830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1dc0a-65a6-4b76-af28-afb592f7cec5 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.828466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1dc0a-65a6-4b76-af28-afb592f7cec5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.828946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1dc0a-65a6-4b76-af28-afb592f7cec5 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.830647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 444/878] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:13.836100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.836639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.837118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.853764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:13.855275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:13.918403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:13.926480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df8f15-9afa-4e83-b209-1ea6200ce9b2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:13.935132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 445/879] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:13 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:13.958320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:13.973036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:13.973849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:13.974260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:13.974852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:13.976224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:13.976616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:13.978368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9e7a4-d1ba-47d7-9cb0-f42d8af6e0ea None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:13.985074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 435/880] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:13 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:14.042084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:14.076122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a79a0-9514-4b6a-b424-aeffe4118db9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:14.113348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:14.114054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.114343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.114621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.115410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:14.116900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095f54d-3be1-4567-97e9-d4347ab777e8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:14.135708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 446/881] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:13 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:14.361497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad67f-5ada-41b8-9aff-4d8db328802c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.361919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad67f-5ada-41b8-9aff-4d8db328802c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ad67f-5ada-41b8-9aff-4d8db328802c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.363857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 436/882] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:14.369259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.369553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.369835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.386464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:14.387691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:14.460124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 447/883] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:14.483465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad67f-5ada-41b8-9aff-4d8db328802c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:14.605498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3ad67f-5ada-41b8-9aff-4d8db328802c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:14.640073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:14.640830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.641121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.641400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.654652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:14.663228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:14.664176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:14.665028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 437/884] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:14.670423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:14.729368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:14.777114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba0e97-c5fc-40c6-b6e8-9da4c0285ebd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:14.813384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:14.814074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.814359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.814638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.815452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:14.815839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:14.817346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:14.826110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 448/885] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:14.832303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:14.855989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:14.940233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a150f50-553d-4258-82d3-1430ccd0f073 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:14.976286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:14.977474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:14.977968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:14.978440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:14.989164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:14.990961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:15.009151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:15.010086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:15.011074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:15.011074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:15.011074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:15.011074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:15.013315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 438/886] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:15.019279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.120038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e4376-5fde-419e-88ec-8a8ae65cf90b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.144475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.160555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.161261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.161548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.161830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.162581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:15.162891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.164345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0a328-a456-4057-8983-c55315b96b0a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.177923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 449/887] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:15.183278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.247826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde913-fffe-4f68-9e95-5e9b9abd70e5 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.248116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde913-fffe-4f68-9e95-5e9b9abd70e5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.248395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dde913-fffe-4f68-9e95-5e9b9abd70e5 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.249904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 450/888] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:15.254998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.255317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.255602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.261823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.272694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:15.273899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:15.298689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bc405-9142-48e2-a0d7-cc2521ab7bba None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.338433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.339622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.339910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.340190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.350046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.351666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.352054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 451/889] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:15.371684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:15.372918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:15.373932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:15.375965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.376367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 439/890] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:15.382237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.477564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e670656-b2cd-4789-9f74-b36fd1b66b2b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.488499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.491813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad863be-cd0f-4213-812e-19f95a28a952 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.513483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.514225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.514510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.514785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.515813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.516919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.525791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:15.528978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:15.530171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:15.531208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:15.535256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 452/891] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:15.535794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.536106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.536804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.538053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.538496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.539768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:15.540245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.541889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.566741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 440/892] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:15.567996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.649715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7d2d5-807b-49a5-8fcf-679e38e2161d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.676717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a29185-6168-477c-8bd6-12e89ff83bf6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.684468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.685157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.685444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.685924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.686429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:15.686742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.688425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.693384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 453/893] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:15.698916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.716067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.716979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.717432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.717942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.740280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.783405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.797597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.799457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.803672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1550ca-d347-4fa1-a2b1-7c7282bff7a9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.838621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.839340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.839628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.839904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.840623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:15.840940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.842328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:15.852383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 454/894] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:15.853258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.914796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 441/895] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:15.916405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:15.955777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b313aa-9968-482b-88ed-f571014a30a6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:15.957457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:15.991615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:15.992295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:15.992619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:15.992857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:15.993667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] RBAC: Adding query filter params (interface=public, service_id=1690adf404f74fe795c3a59430b368b0, region_id=RegionOne) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:15.993984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:15.995969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.007309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 455/896] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/endpoints?service_id=1690adf404f74fe795c3a59430b368b0&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:16.009288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.017868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.029972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f3b59-e2f9-4914-9f22-9ba30392f457 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.068740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.069912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.070385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.070801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.082414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.084161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:16.104064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:16.105180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:16.106372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:16.106372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:16.106372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:16.106372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:16.108259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 442/897] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:16.114030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.116706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05074c-2dd7-4c6e-bc8e-4e4624860966 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.151125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.151839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.152097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.152378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.153070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.154451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aab8c0-8356-4dd9-90b3-60e38adf5c7f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.167126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 456/898] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:03:16.167126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5035-d4fd-4d1b-ac8b-6e3a6665d50a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.167126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5035-d4fd-4d1b-ac8b-6e3a6665d50a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.167126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd5035-d4fd-4d1b-ac8b-6e3a6665d50a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.168293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 457/899] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:16.169765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.170056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.170337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.189360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:16.190656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:16.223541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.251344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83237c-5d77-4ac0-8e77-d954315cea56 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.270681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 458/900] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:16.294568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.296733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.297667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.298148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.298623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.299659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:16.300194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.302382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462c58a-af3f-4dcd-a033-01fc41165328 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.316013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 443/901] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:16.319800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15e479-d04e-4793-a3d9-97f9301362fe None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.319985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15e479-d04e-4793-a3d9-97f9301362fe None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.320348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15e479-d04e-4793-a3d9-97f9301362fe None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.321900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 444/902] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:16.322643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15e479-d04e-4793-a3d9-97f9301362fe None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.327499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.419080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ac757-235a-472a-8dbc-f8bd28bba9fd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.438084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15e479-d04e-4793-a3d9-97f9301362fe None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.456909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.457595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.457879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.458159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.458941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.460435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d73fbe-9ac9-4986-a957-c938890e3c3a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.475684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 459/903] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:16 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:16.477200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.477661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.478133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.484171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.485374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.485923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.486368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.496126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.498365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.502480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:16.504419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:16.516098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:16.517067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:16.517966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:16.526083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.526920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 445/904] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:16.528110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.605589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.609892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 460/905] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:16.638003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.667729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713c12-82ef-4043-b3b1-de39de3d0933 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.710141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.710824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.711188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.711475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.712402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:16.712577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.715252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.726310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 446/906] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:16.732165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.751950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53deeacf-c615-43fc-9780-1bde381f3e00 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.788735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.790232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.790232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.790232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.790768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.792201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:16.805389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:16.806395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:16.807270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 461/907] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:16.812848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:16.829493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.866858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299b52-68aa-45a9-8a17-51f09667692d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.885204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:16.903740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.904416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.904702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.904982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.905829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] RBAC: Adding query filter params (role.id=b2f31e54cd054b9aa89d6df02c6b7ec4, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=8079221b336646d3af1dc7aba9719ad0) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:16.906150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.907640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.919412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 447/908] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/role_assignments?user.id=8079221b336646d3af1dc7aba9719ad0&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:16.921096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f082d4ed-a3f5-4dd4-9173-a7e3312461a3 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:16.961563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:16.962264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:16.962554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:16.962836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:16.963601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:16.963918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:16.965359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:16.971734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 462/909] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:16.978331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:17.098416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ea988-8316-4449-b3fb-4682fae10c7a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:17.133417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:17.134014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:17.134295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:17.134569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:17.135313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:17.135619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:17.137000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:17.142384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 448/910] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:17.148539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:17.228922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:17.253839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741068e-a901-4af7-943c-21df27913e9c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:17.288386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:17.289080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:17.289373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:17.289648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:17.290461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] RBAC: Adding query filter params (interface=public, service_id=e52dc101eb15408e8f110cd9c21e8e9c, region_id=RegionOne) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:17.290782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:17.292247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390f494-a55b-4e13-b5d4-5098cc1a7814 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:17.300275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 463/911] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:17 2022] GET /identity/v3/endpoints?service_id=e52dc101eb15408e8f110cd9c21e8e9c&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:17.306898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:17.355986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:17.414656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c68b2-2b16-4287-b78e-3c1a1fe2e281 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:17.449852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:17.450890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:17.451393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:17.451859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:17.452765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:17.454390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55eeec-42b0-4403-893b-2c7a701e338a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:17.467307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 449/912] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:17 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:03:18.260958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a680-92e7-423e-a901-256b8774cf3f None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.261519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a680-92e7-423e-a901-256b8774cf3f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.261519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1a680-92e7-423e-a901-256b8774cf3f None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.263176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 464/913] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:18.268916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.269358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.269433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.286899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:18.288150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:18.356385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 450/914] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:18.380440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1a680-92e7-423e-a901-256b8774cf3f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:18.493820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1a680-92e7-423e-a901-256b8774cf3f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:18.529001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:18.529739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.530025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.530306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.530970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:18.532391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d58520a7-a834-4aa0-9e94-d5d7d78cb76a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:18.544591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:18.545646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.546898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 465/915] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:18.552924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:18.608678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:18.650303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36b98-6d05-4065-b1f0-c91db7ae67e8 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.650628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36b98-6d05-4065-b1f0-c91db7ae67e8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.650877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b36b98-6d05-4065-b1f0-c91db7ae67e8 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.652543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 466/916] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:18.657885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.658175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.658459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.670774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:18.676853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:18.678059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:18.684216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c4eef-60f6-48fc-a0c9-5795ec827aec None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:18.720096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:18.720905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.721193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.721472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.722438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:18.722750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:18.724267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:18.729813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 451/917] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:18.735085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:18.763105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 467/918] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:18.780846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:18.839351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898022b-74a8-4741-a010-ada2f21914bf None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:18.847527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:18.876314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:18.877140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.877517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.877993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.878887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:18.879324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:18.880810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:18.884632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a64f6b-4cf7-4235-9c20-8a42adcf5bc2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:18.886492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 452/919] 10.222.0.13 () {56 vars in 1114 bytes} [Thu Jun 30 04:03:18 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:18.896327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:18.925946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:18.926780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:18.927067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:18.927381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:18.928046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:18.929451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:18.932119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:18.942441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:18.943605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:18.944436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:18.948878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 468/920] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:18.954939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.013792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d120a-d70f-4f0e-b4fe-4fcd6035a3e1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.052133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.052734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.053161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.053451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.054211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.056266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb55822d-c253-4ecd-b6e6-3d98da14e6e7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.080722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583d45-59b3-482d-8fc3-b99117e56e2c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.081101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 453/921] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:19.082522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a8cf6-93af-4ecb-9435-a26a7b69fa84 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.082852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a8cf6-93af-4ecb-9435-a26a7b69fa84 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.083121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a8cf6-93af-4ecb-9435-a26a7b69fa84 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.088741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 454/922] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:19.090434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.090731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.091013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.113504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:19.116645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:19.139606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.154704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.156121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.157093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.157588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.158862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:19.159569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.162097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.170116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 469/923] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:19.177254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.212927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 455/924] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:19.236072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.257331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.311252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e917d-ea63-48d5-a0dc-6f986aed54ec None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.347953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.349019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.349522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.349995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.350927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:19.351452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.353051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.358779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 470/925] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:19.365143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.366824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4bd19e-8f74-4d9d-9152-066967c820c2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.407085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.408332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.408814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.409417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.423865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:19.436710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:19.437705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:19.438568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:19.440659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 456/926] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:19.447529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.464122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.482972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304d9f-db30-4f82-9115-ba8bb80b4ef4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.508362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.523260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.523837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.524120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.524396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.525637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] RBAC: Adding query filter params (interface=public, service_id=374f3a9c39964154aff52765a807a3d4, region_id=RegionOne) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:19.525637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.526964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.537387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 471/927] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/endpoints?service_id=374f3a9c39964154aff52765a807a3d4&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:19.540437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.572361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8699d-ec54-49be-9799-fb1d377ce8bb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.609922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.610917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.611411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.611693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.612504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:19.613070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.614294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914eaa15-6334-452f-95e4-94cdfd6c2615 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.628908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 457/928] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:19.628908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83780acf-742f-479d-b566-29ff2e38b6a5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.628908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83780acf-742f-479d-b566-29ff2e38b6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.628908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83780acf-742f-479d-b566-29ff2e38b6a5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.631451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 458/929] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:19.632164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83780acf-742f-479d-b566-29ff2e38b6a5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.659681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ba76f-82ec-4cd4-b7d1-6f74d0d1fcfa None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.701288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.701969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.702255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.702536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.703253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.704632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430d821-a16b-473b-a2c8-3a9f80b6e9ee None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.711652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 472/930] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:03:19.712713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.712999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.713508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.716329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.742544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:19.744821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:19.746932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83780acf-742f-479d-b566-29ff2e38b6a5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.795702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:19.796708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:19.796996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:19.797266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:19.810613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:19.812215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:19.821412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:19.833481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:19.834482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:19.835421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:19.835421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:19.835421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:19.835421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:19.839953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 459/931] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:19.841467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.854218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 473/932] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:19.876925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:19.968040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8338c-e535-4af7-93fe-80350b082cbc None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:19.999620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed41737-6e69-4fc6-909f-6d5a67180249 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.004117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.014922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.016445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.017193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.017675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.018984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:20.019571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.021625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.038163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.038497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 460/933] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:20.039928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.040420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.040916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.041777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.043403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.043963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:20.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:20.058452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 04:03:20.059362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:20.061900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 474/934] 10.222.0.13 () {56 vars in 1132 bytes} [Thu Jun 30 04:03:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:20.067543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.132361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.155023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451dc285-a860-405a-8e1f-f86a3ca17b9b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.195852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.197041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.197388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.197998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.204568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34b53-f495-4bd9-891c-a52dd13d794e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.205807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.208380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:20.226870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:20.228107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:20.229323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:20.234727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 461/935] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:20.235963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.248447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.249684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.250416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.250853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.251986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:20.252471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.254556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.261360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 475/936] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:20.266821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.297515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.358880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ba04d-7307-4597-b18a-a35e1e7d6bf3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.390807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b629a-daae-499d-ad89-48c37d7b45a9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.396841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.398129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.398620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.399120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.400146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:20.400634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.402315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.403697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.423386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 462/937] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:20.430597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.435839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.436781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.437310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.437788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.438715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:20.439781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.442265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.449950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 476/938] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:20.455114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.566770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fcd4-63a9-4ab8-928a-bf22265558e2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.571935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b745f3-04a8-4485-8228-525c572e8d96 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.584749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.611556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.612265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.612545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.612814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.613557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.614933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.624426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.625274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.625579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.625866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.637489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.638986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.647645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 477/939] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:20.657546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:20.658570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:20.659673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:20.659673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:20.659673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:20.659673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:20.659673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 463/940] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:20.665731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.712567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:20.799203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69803fd-1bf0-47ec-8a85-7037fbcc69a2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.837573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:20.838258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:20.838541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:20.838815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:20.839580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:20.839890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:20.841274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:20.854010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 478/941] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:20.859699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:20.965143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335003-0e4b-47ec-bc2d-5b80764f86ea None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:20.999901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:21.000759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.001044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.001321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.008315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:21.009772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:21.023829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:21.024809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.025931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 464/942] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:21.032790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:21.131712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:21.147997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795d70-c659-4293-aafb-81476af8b804 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:21.183564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:21.184244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.184530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.184806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.185612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:21.185925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:21.187387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc077f-0d04-4147-82a7-e91bd31f5d22 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:21.200166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 479/943] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:21.206206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:21.227212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:21.311776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5ae0f-56a1-498e-bb5c-bcdd131ff2f0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:21.346166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:21.347237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.347542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.347820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/8079221b336646d3af1dc7aba9719ad0/roles/b2f31e54cd054b9aa89d6df02c6b7ec4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.350028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=8079221b336646d3af1dc7aba9719ad0, role_id=b2f31e54cd054b9aa89d6df02c6b7ec4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:21.351519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:21.355928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e02dc0-1733-4845-ac03-6019c32428f5 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.356319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e02dc0-1733-4845-ac03-6019c32428f5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.356501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e02dc0-1733-4845-ac03-6019c32428f5 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.357953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 480/944] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:21.363313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.363954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.364562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.369622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 465/945] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:21 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/8079221b336646d3af1dc7aba9719ad0/roles/b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:03:21.383182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:21.384839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:21.460756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 481/946] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:21.486023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:21.576179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:21.605013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0879-4764-45c6-93b1-d1e6dac93b3c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:21.640263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:21.641333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.641822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.642297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.643239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:21.644925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:21.646514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING keystone.server.flask.common [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 04:03:21.666236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.sql.core [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 04:03:21.666236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 04:03:21.666236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [parameters: {'id': '27ce1f62ced84ba2aa1066401b258825', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 04:03:21.666236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80703) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27ce1f62ced84ba2aa1066401b258825.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27ce1f62ced84ba2aa1066401b258825. Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:03:21.684689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result.read() Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 04:03:21.685580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application yield resource Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise self.value Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application session.commit() Jun 30 04:03:21.686854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.session.flush() Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application compat.raise_( Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 04:03:21.687808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 04:03:21.688732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result.read() Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:03:21.689550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [parameters: {'id': '27ce1f62ced84ba2aa1066401b258825', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.690529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:21.692865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 27ce1f62ced84ba2aa1066401b258825. Jun 30 04:03:21.695082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:21.698184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 466/947] 10.222.0.13 () {60 vars in 1140 bytes} [Thu Jun 30 04:03:21 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:21.703845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:21.790999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:21.815466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f0b9f-e09c-43d8-9a8a-8d5d0b702db7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:21.852074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:21.852889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.853167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.853442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.860454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:21.861930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-7f4e601c-9ac5-4a24-86b8-bd600ec0f4d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:21.880662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:21.882896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:21.885066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 482/948] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:21.887965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:21.911627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de928914-cf52-4360-8d29-2d69e84fa803 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.911954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de928914-cf52-4360-8d29-2d69e84fa803 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.912190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de928914-cf52-4360-8d29-2d69e84fa803 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.913803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 483/949] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:21.922106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:21.922106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:21.922106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:21.940493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:21.942739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:22.015267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325b3ccb-74a5-4402-8b28-a690c03ab283 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:22.031134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 484/950] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:22.047197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:22.062117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:22.062924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.063348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.063720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.064582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:22.064985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:22.067022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:22.068968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:22.090248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 467/951] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:22.127342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:22.165552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4935f11-6c45-4bda-be36-7b710fd6305d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:22.203938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:22.204786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.205073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.205345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.206114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:22.207640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:22.220566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:22.221842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:22.223003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 485/952] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:22.228440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:22.354948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef03d2b-a91f-44c1-af0a-e90c514af572 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:22.389220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:22.389912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.390202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.390476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.391231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:22.391542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:22.392933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:22.398843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 468/953] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:22.404670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:22.508334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e605c64-a75d-416b-a409-16c7c3c852f6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:22.537935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:22.543958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:22.544653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.544937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.545223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.546024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:22.546723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:22.548222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf1b53-ec40-4687-888e-2591cdeeeea3 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:22.554468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 486/954] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:22.560123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:22.630321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:22.673493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e380843-62f6-4ba1-a5b6-4778c1c22871 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:22.708549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:22.709228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.709516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.709795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.710555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:22.712025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180ce1a-008c-4d91-ab13-b51c206ca3d0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:22.736186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 469/955] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:22.852791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb238-204c-4a50-9564-e769d731cfcb None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.853139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb238-204c-4a50-9564-e769d731cfcb None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.853364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb238-204c-4a50-9564-e769d731cfcb None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.855119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 487/956] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:22.861017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:22.861367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:22.861653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:22.882474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:22.884502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:22.964471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 470/957] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:22.988393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bb238-204c-4a50-9564-e769d731cfcb None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:23.092164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bb238-204c-4a50-9564-e769d731cfcb None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:23.130165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:23.130986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.131307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.131591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.132256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:23.133696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:23.145761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:23.146874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:23.147778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 488/958] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:23.155400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:23.205234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:23.264142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65c0a1-8cc5-4c22-b856-bbb39a603f1d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:23.299079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:23.299849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.300130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.300774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.301500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:23.301805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:23.303211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5653f5-1c1d-4073-ad66-01e7f5c547f8 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:23.308432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 471/959] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:23.316320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:23.376842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:23.422240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0f8f-32a3-40a5-8478-7edd0057865d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:23.457422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:23.458147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.458436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.458715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.459463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:23.459776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:23.461175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e8c0-6e97-4c02-afd6-6c640e6befcb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:23.466513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 489/960] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:23.556681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1bc68-78eb-4812-8afc-276c50a7d03c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.557048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1bc68-78eb-4812-8afc-276c50a7d03c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.557239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1bc68-78eb-4812-8afc-276c50a7d03c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.558922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 472/961] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:23.570513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.571416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.571837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.596331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:23.598068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:23.674529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 490/962] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:23.698837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1bc68-78eb-4812-8afc-276c50a7d03c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:23.806200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1bc68-78eb-4812-8afc-276c50a7d03c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:23.809395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:23.843010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:23.843864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:23.844148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:23.844424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:23.859523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:23.868115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:23.869071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:23.870331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 473/963] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:23.877314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:23.932060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:23.981000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39a79a-7b8d-4acf-9a65-6cd3c1d26a99 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.015758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.016454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.016737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.017013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.017785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:24.018094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.019823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.030296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 491/964] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:24.036525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.146341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f4d1-474b-4cd6-9edd-1bc659c52cd1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.181275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.182142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.182370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.182647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.194030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.195532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-05ca5ad2-0ada-46f3-ae5e-d2f8ad8c2d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:24.213554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:24.214533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:24.215857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:24.215857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:24.215857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:24.215857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.215857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 474/965] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:24.224294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.329380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf9aa1-e290-456f-b857-5d4c8080ef31 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.339039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223dce1-78e1-46b9-8a18-a63e61021d1b None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.339357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223dce1-78e1-46b9-8a18-a63e61021d1b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.339642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223dce1-78e1-46b9-8a18-a63e61021d1b None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.341196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 475/966] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:24.347130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.349179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.350821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.350821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.368493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.369157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.369436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.369707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.370427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:24.370732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.372298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.373193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:24.375386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:24.388303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.389268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 492/967] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:24.390631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.479117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 476/968] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:24.495386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.506164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9b1a6-aa0b-489f-a215-ea20813338db None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.554292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.556050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.556513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.557226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.566597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.568413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:24.591082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:24.592734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 04:03:24.593914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:24.599681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 493/969] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:24.599681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.613138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd190ba-aecd-4375-801e-a0519916f63d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.642316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.652923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.653982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.654466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.654950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.655893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.657531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.671976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.sql.core [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 04:03:24.671976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 04:03:24.671976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [parameters: {'user_id': '4d4dd6b5d07d4ad4a3c08939d5275191', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 04:03:24.671976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80703) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result.read() Jun 30 04:03:24.681642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:03:24.682554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application yield resource Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application next(self.gen) Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise self.value Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application session.commit() Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 04:03:24.683601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.session.flush() Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application compat.raise_( Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 04:03:24.684472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 04:03:24.685384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application conn.query(q) Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application result.read() Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 04:03:24.687454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [parameters: {'user_id': '4d4dd6b5d07d4ad4a3c08939d5275191', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:24.689396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 04:03:24.691673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 04:03:24.693815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.696811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 477/970] 10.222.0.13 () {60 vars in 1131 bytes} [Thu Jun 30 04:03:24 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:24.702729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.719699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51971f0-8f56-4a01-b106-38c7efd961aa None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.720749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.757602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.758205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.758491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.758770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.759660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:24.759973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.761467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.785536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 494/971] 10.222.0.13 () {56 vars in 1149 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:24.786621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.830095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a05c1-fcb2-4749-9196-66ab413cd8af None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.866910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.868111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.868663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.869259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.878141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.879050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.881084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:24.888540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec344c-921f-4c39-9b3f-6fdd2dea64d3 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:24.898148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:24.899140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 04:03:24.900108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:24.907361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 478/972] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:24.908538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:24.926029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:24.926689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:24.926966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:24.927579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:24.982982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:24.996780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:24.998306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a1f59-2640-40b6-8927-ae9f9165656b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.023420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898699d-2f9f-403f-81d7-4fc2b5bea1f2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.059954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.060737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.061221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.061417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.062214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.062529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.064033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d078d4-fba5-4354-8e89-2400b1ee097a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.090546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.104855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 495/973] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:25.104855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddd6d8-28ef-4bb7-8807-13bd9215530a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.104855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddd6d8-28ef-4bb7-8807-13bd9215530a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.104855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddd6d8-28ef-4bb7-8807-13bd9215530a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.104855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 496/974] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:25.105741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddd6d8-28ef-4bb7-8807-13bd9215530a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.108877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 479/975] 10.222.0.13 () {56 vars in 1101 bytes} [Thu Jun 30 04:03:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:25.110566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.110865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.111179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.130808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:25.132589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:25.215125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 480/976] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:25.239150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.248182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddd6d8-28ef-4bb7-8807-13bd9215530a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.300218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.301329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.301624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.301904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.304628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.314866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.316872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:25.338245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:25.339264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:25.340259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:25.340259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:25.340259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:25.340259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:25.343225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 497/977] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:25.345429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.368099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d536a7-537a-4a7b-aae8-198511fede78 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.400534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.415533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.416363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.416653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.416935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.417724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.419261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:25.432392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:25.433466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:25.434326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:25.438706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 481/978] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:25.439528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.462200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b931f6-b898-4888-b00d-8c006e20982c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.504432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.505113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.505655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.505944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.507311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.507625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.509564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.522627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 498/979] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:25.528024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.542484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4547c3-17f6-437f-bb73-472c89e13b64 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.563347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.581252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.582792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.583540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.584011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.585099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.585709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.587143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac0b37-0ce1-4462-9933-1cef3af9722c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.592304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 482/980] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:25.593551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b52-616d-4c7d-92ec-838959455104 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.593845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b52-616d-4c7d-92ec-838959455104 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.594125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b52-616d-4c7d-92ec-838959455104 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.595512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 483/981] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:25.597812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0b52-616d-4c7d-92ec-838959455104 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.636640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6cab6-9dbc-4d56-a765-2462430b4bb6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.638894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.671559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.672362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.672645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.672912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.682085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.683667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e46a8df8-edaa-41a0-aa34-5984cdb055ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:25.697952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:25.698869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:25.699805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 499/982] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:25.700496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.700638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.700967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.705580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0b52-616d-4c7d-92ec-838959455104 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.719042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:25.720247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:25.739819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.740492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.740780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.741063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.741773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.742084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.743482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.748239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 484/983] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:25.749005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.788063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 500/984] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:25.788893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.796316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.854541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121bc2c-a530-4a86-99e1-b59827197a79 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.881254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:25.888256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aa5d6-66ea-4710-996d-bce00cc1a70f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:25.888574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.889234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.889519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.889789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.890562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.890861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.892344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.904258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 485/985] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:25.905026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:25.923488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:25.924146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:25.924434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:25.924912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:25.926098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] RBAC: Adding query filter params (interface=admin, service_id=374f3a9c39964154aff52765a807a3d4, region_id=RegionOne) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:25.926411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:25.927837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:25.932967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 501/986] 10.222.0.13 () {56 vars in 1249 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/endpoints?service_id=374f3a9c39964154aff52765a807a3d4&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:25.933755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:26.003964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879127d-9ea0-41a6-996a-2df45b37e4a6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:26.013221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:26.033237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95d191-e15f-4b50-94ec-bd25a57ad66f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:26.040408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:26.041083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:26.041372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:26.041652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:26.042382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:26.043805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:26.059361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 486/987] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:25 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:26.060163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:26.080855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:26.081652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:26.081912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:26.082211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:26.083089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] RBAC: Adding query filter params (role.id=b2f31e54cd054b9aa89d6df02c6b7ec4, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=8079221b336646d3af1dc7aba9719ad0) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:26.083407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:26.084909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8167f-39af-4b21-967f-d8f5c8b58e19 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:26.093430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 502/988] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:25 2022] GET /identity/v3/role_assignments?user.id=8079221b336646d3af1dc7aba9719ad0&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:26.187731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33dc70-dc46-4d40-af81-035076b6f1f9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:26.227824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:26.228543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:26.228828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:26.229755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:26.229862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:26.231246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39820c8-79b9-401b-841a-9f42f6bcdc98 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:26.239188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 487/989] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:26 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:03:28.105885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1a8b-b669-436a-a0f4-b78ae44520ca None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.106433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1a8b-b669-436a-a0f4-b78ae44520ca None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.106433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b1a8b-b669-436a-a0f4-b78ae44520ca None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.108001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 503/990] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:28.113196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.113656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.113733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.130658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:28.131871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:28.159796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.171280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1794-d1b6-49da-a85d-793a48f8b543 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.171579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1794-d1b6-49da-a85d-793a48f8b543 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.171873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb1794-d1b6-49da-a85d-793a48f8b543 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.173258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 504/991] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:28.178572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.178862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.179164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.197937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:28.199190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:28.208082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 488/992] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:28.213916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.230727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.272051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 505/993] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:28.293840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.337176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f581b1-6de1-42ff-bbb8-173c980fbdfd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.372202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.372936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.373218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.373494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.374149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.375568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-891bab7a-fba7-49f0-9285-7fc47e954c90 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:28.387720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:28.388823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:28.389696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 489/994] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:28.390500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d82b78-efb8-4602-bd44-6252e2d46dbd None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.390500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d82b78-efb8-4602-bd44-6252e2d46dbd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.390629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d82b78-efb8-4602-bd44-6252e2d46dbd None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.392066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 490/995] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:28.395930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6afac-5e12-4274-a0e4-760d98d19b67 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.396099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d82b78-efb8-4602-bd44-6252e2d46dbd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.427288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.434924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.435768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.436050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.436333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] PATH_INFO: `/v3/services/image` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.436968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.438363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b050d448-8316-49a6-a8aa-c47d03f2a047 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:28.450365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:28.451444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.452298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 506/996] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:28.453635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.453635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.453798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.470703 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:28.471994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:28.480003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.504003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d82b78-efb8-4602-bd44-6252e2d46dbd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.536803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.537477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.537766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.538050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.538766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:28.539076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.539610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 507/997] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:28.540444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.540546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.545777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 491/998] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:28.551723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.648350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.648781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941266a1-fc89-4e49-82d0-67f6373555fd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.658467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b5482-57fc-45dc-a361-8fa5e8d2f49a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.684986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.685661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.685944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.686227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.686932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] RBAC: Adding query filter params (name=image) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:28.687271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.688689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.693514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 508/999] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:28.694282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.694387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.695088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.695394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.695669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.696372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] RBAC: Adding query filter params (type=compute) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:28.696677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.698235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.707891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 492/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:28.707891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.708129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.797313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495e544-2f35-4692-aeda-52075f6f73ee None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.804758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a6b-aa51-408f-8268-e92c06f78dee None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.830504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.831418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.831695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.831969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.832632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.834013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.837463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:28.838125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:28.838414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:28.838693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:28.839437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] RBAC: Adding query filter params (type=image) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:28.839743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:28.841094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:28.845664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 493/1001] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:28.845850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:28.846958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 04:03:28.847989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:28.848722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.848794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 509/1002] 10.222.0.13 () {56 vars in 1126 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:28.853327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:28.874490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.934998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:28.970857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b366a8-055b-42a7-ad4f-779ea5064080 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:28.971137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95427bc2-8f8c-46f3-81f4-d1278f4c91cd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:29.009117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:29.009796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:29.010077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:29.010353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:29.011181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] RBAC: Adding query filter params (interface=public, service_id=7ac126e72cfc4612a99c0d170e84ad2e, region_id=RegionOne) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:29.011497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:29.012113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:29.012814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:29.012928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:29.013087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:29.013370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:29.014084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:29.014389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:29.015791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:29.022368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 510/1003] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:29.024805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 494/1004] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:28 2022] GET /identity/v3/endpoints?service_id=7ac126e72cfc4612a99c0d170e84ad2e&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:29.025715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:29.027983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:29.116751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:29.136322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46bfa-fd31-4008-a949-944ed3b1fa9b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:29.145871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19387818-6177-45a5-8add-cbcf086e579f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:29.174158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:29.174883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:29.175742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:29.175848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:29.176508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:29.177009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:29.178202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:29.183380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 495/1005] 10.222.0.13 () {56 vars in 1122 bytes} [Thu Jun 30 04:03:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:29.186835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:29.187880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:29.188328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:29.188800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:29.189001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:29.189893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:29.192135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6deb7-1909-4bb3-9845-ee1591f3d2e4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:29.194645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:29.215814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 511/1006] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:29 2022] GET /identity/v3/services => generated 1727 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:03:29.301167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf289a2d-5bff-43b5-8421-cec3cb4ac034 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:29.339401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:29.340093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:29.340381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:29.340662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:29.341427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:29.342845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45cb46-a80b-437d-bfd9-a40fb7662fa5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:29.360879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 496/1007] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:30.866725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226e82c-a627-4919-9d03-563a91f4a39e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:30.867262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226e82c-a627-4919-9d03-563a91f4a39e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:30.867262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226e82c-a627-4919-9d03-563a91f4a39e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:30.868680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 512/1008] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:30.873920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ef4a1-de72-442f-bf9c-5b092b8fb29a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:30.874360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ef4a1-de72-442f-bf9c-5b092b8fb29a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:30.874437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ef4a1-de72-442f-bf9c-5b092b8fb29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:30.887053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:30.892187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-142ef4a1-de72-442f-bf9c-5b092b8fb29a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:30.894036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-142ef4a1-de72-442f-bf9c-5b092b8fb29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:30.970284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 497/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:30 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:30.992663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226e82c-a627-4919-9d03-563a91f4a39e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.097925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226e82c-a627-4919-9d03-563a91f4a39e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.134100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.134744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.135027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.135380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.136122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.137492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d900c-2394-4508-9b22-70fe6ee3fe20 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.153305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 513/1010] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:30 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:31.250555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2fc2f-b92d-4172-81ab-f0d37f81c1cc None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.250821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2fc2f-b92d-4172-81ab-f0d37f81c1cc None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.251134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2fc2f-b92d-4172-81ab-f0d37f81c1cc None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.253290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 498/1011] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:31.258902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90885668-4cb2-449b-ac36-416e33a6d99c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.259297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90885668-4cb2-449b-ac36-416e33a6d99c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.259468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90885668-4cb2-449b-ac36-416e33a6d99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.278967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-90885668-4cb2-449b-ac36-416e33a6d99c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:31.280230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-90885668-4cb2-449b-ac36-416e33a6d99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:31.341902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:31.364277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 514/1012] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:31.387237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2fc2f-b92d-4172-81ab-f0d37f81c1cc None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.430427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4b43-d2dc-41db-bfc6-ff72ab5b977c None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.430752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4b43-d2dc-41db-bfc6-ff72ab5b977c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.431002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc4b43-d2dc-41db-bfc6-ff72ab5b977c None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.432597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 515/1013] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:31.438253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.438663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.439073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.461378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:31.462591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:31.505439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2fc2f-b92d-4172-81ab-f0d37f81c1cc None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.533247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 516/1014] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:31 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:31.537543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:31.543338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.544627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.545281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.545565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.546290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.547730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.557883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:31.560587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:31.561618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:31.562487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 499/1015] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:31.569285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.666098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42752e1-123c-44c5-a4b1-9976e1fb552e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.681442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:31.682439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f6479-748f-447e-b02f-5241cb3661a3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.705034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.706089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.706386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.706647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.707940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.709309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.711043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 517/1016] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:31.716566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.718831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.719550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.719836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.720119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.720832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] RBAC: Adding query filter params (name=compute) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:31.721145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.722551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.728164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 500/1017] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:31.733473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.796521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:31.826159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235d1ca-017a-42dc-b1d1-2f7121d9bdc9 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.841217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433261b-141b-4b29-959a-8c41ed7713fe None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:31.863230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.863912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.864307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.864587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.865416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.867073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d420c2-947e-43a2-a079-d35dcf9b21d7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.875233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:31.875918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:31.876207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:31.876487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:31.877205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:31.877748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:31.879024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:31.883927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 501/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Thu Jun 30 04:03:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:31.889646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:31.902541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 518/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:31 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:03:32.007497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ecc06-9027-452c-b0e5-881460fcc791 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:32.033600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:32.046563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:32.047480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:32.047774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:32.048053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:32.048883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:32.051177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bc0971-d697-4923-bd81-6fe555b18090 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:32.075052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 502/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:31 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:33.197939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e85ea-5adb-4b10-b9c7-5062afb2019d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.198528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e85ea-5adb-4b10-b9c7-5062afb2019d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.198528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e85ea-5adb-4b10-b9c7-5062afb2019d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.199881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 519/1021] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:33.205411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.205937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.206013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.223641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:33.224794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:33.300642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 503/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:33.322830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e85ea-5adb-4b10-b9c7-5062afb2019d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:33.359938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:33.428644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e85ea-5adb-4b10-b9c7-5062afb2019d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:33.462159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:33.462954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.463279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.463553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] PATH_INFO: `/v3/services/image` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.464205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:33.465584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:33.477199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:33.478191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:33.479039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:33.483419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 520/1023] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 30 04:03:33 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:33.484429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:33.596737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222cec1-acf3-49a9-9aed-1a799ec838bc None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:33.604686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:33.635597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:33.636588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.637018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.637440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.638316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] RBAC: Adding query filter params (name=image) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:33.638785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:33.640344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:33.645726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 504/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:33.651359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:33.753048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:33.757536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d4a1-80bd-49ed-b4ab-e85391ae5c47 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:33.793242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:33.793920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.794203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.794481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.795227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] RBAC: Adding query filter params (type=image) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:33.795541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:33.796975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:33.802743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 521/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:33.808594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:33.913572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3f8f2-378b-44ac-ae39-c3891a791c40 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:33.949901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:33.950573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:33.950856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:33.951143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:33.951951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] RBAC: Adding query filter params (interface=public, service_id=d920bb10b42e4ec5a944322a9ca4f111, region_id=RegionOne) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:33.952258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:33.953634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84684818-0f0b-42e7-bfaf-9c86dfddaa72 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:33.959174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 505/1026] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:03:33 2022] GET /identity/v3/endpoints?service_id=d920bb10b42e4ec5a944322a9ca4f111&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:33.964575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.011882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bee729-0007-4af2-958c-688b4fc3bdf5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.012081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bee729-0007-4af2-958c-688b4fc3bdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.012354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bee729-0007-4af2-958c-688b4fc3bdf5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.013907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 506/1027] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:34.018900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.019228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.019499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.036252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:34.037474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:34.064739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:34.066520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ab448-9ca8-471b-8e7e-8ee1865c08db None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.100950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.101621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.101902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.102175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.102857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:34.104269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faa4dc2-4a91-4c38-a3b9-9c463b512d31 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:34.108388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 507/1028] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:34.119129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 522/1029] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:03:33 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:03:34.133772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.233081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8544-fc94-4921-8d64-cbb06227cdee None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.233673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8544-fc94-4921-8d64-cbb06227cdee None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.233673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a8544-fc94-4921-8d64-cbb06227cdee None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.234982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 523/1030] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:34.241093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.241383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.241663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.245319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586e63a-4b3e-4055-bac8-eaf2fb992e91 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.260047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:34.261470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:34.286482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:34.289691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.290515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.290795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.291063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.292320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:34.293817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:34.295502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 508/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:34.302047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.335794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:34.343128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 524/1032] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:34.366992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.413031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de594684-b55d-45c2-a455-48c945098cb4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.448712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.449781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.450293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.450491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.451260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:34.452716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:34.473630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaaf85-e893-4052-9968-176ea6b8a6e8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.474696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 509/1033] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:03:34 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:34.521257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.522085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.522366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.522643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.535085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:34.543704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:34.544838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:34.551840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 525/1034] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:34.554006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.670178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:34.685565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fad46-3e2b-4763-afdf-56695ee93778 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.721814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.722502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.722789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.723062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.723911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:34.724216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:34.725606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:34.733980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 510/1035] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:34.740011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:34.759636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:34.846803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393fa5c-a50f-4c84-aae7-8e34b709ad02 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:34.880752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:34.881560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:34.881847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:34.882119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:34.893127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:34.894606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:34.911904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:34.912854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:34.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:34.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:34.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:34.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:34.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 526/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:34.919966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.023272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31719936-a2ad-46db-b55a-085c482b9c1c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:35.057065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:35.057739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:35.058022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:35.058298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:35.059021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:35.059401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:35.060724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:35.072882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 511/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:35.078382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.144787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:35.180261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeea4d-7e5d-44d3-9281-0bbd467f8f27 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:35.213829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:35.214635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:35.214920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:35.215223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:35.223667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:35.225135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:35.239769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:35.241195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:35.242178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:35.243393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 527/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:03:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:35.246814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.260718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:35.351559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993577d3-e6b0-446a-b872-4b69938ebab7 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:35.384980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:35.385658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:35.385941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:35.386215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:35.387055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:35.387393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:35.388838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:35.406211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 512/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:03:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:35.411911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.517563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b575f-b02f-4e5e-8c65-831099cc78e0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:35.544368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:35.553860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:35.554527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:35.554810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:35.555087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:35.632373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:35.633824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:35.737213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 528/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:35 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:35.742813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.809212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:35.843251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728758f-8483-4a5c-b5ca-26fd8d3f9574 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:35.877037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:35.877894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:35.878309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:35.878586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:35.888857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:35.890303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:35.907738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:35.908730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:35.909895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:35.909895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:35.909895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:35.909895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:35.911478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 513/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:35.914817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:35.943779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.021063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638eb2f1-94f1-4a1c-b9e9-0ddb47ad2b0a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.056179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.056850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.057130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.057402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.058129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:36.058434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.059830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e3b9af-21a1-4ae1-ab6b-e047b6b86a71 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.071636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 529/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:36.077121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.179836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499b999-86b1-4f54-8bc0-e95d8e7f0ec3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.199629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba03cd-9378-4b08-8c5f-813181a0bae1 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.200268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba03cd-9378-4b08-8c5f-813181a0bae1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.200268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba03cd-9378-4b08-8c5f-813181a0bae1 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.200554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.201697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 530/1043] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:36.207448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.207739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.208019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.217091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.217912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.218188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.218457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.224725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:36.225925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:36.226114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.227551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:36.241948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:36.242885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:36.243975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:36.248176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 514/1044] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:36.248891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.303900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 531/1045] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:36.327061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.345867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.368722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa81882-4a48-49be-810b-5a5b5daef824 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.405379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.406427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.406957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.407436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.408417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:36.409257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.411011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.425976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 515/1046] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:36.429396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6f1a-77f7-4c0d-9cab-d24eb950f385 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.431802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.466099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.467025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.467559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.467846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] PATH_INFO: `/v3/services/image` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.468528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.469971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.475196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:36.483469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:36.484479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:36.485328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 532/1047] 10.222.0.13 () {56 vars in 1105 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:36.490879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.538966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edadb96-c4c6-41f3-a4db-632624802a3d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.578022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.578726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.579035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.579320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.580046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.580164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] RBAC: Adding query filter params (role.id=b2f31e54cd054b9aa89d6df02c6b7ec4, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:36.580495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.582086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901e22a-92c9-40a9-920f-a8ff98d192da None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.587768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 516/1048] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:36.589045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8eeb-0f79-43b1-a4e8-9e47369756a2 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.589333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8eeb-0f79-43b1-a4e8-9e47369756a2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.589605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d8eeb-0f79-43b1-a4e8-9e47369756a2 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.590840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 517/1049] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:36.596925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.597223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.597495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.613510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96726-10f1-417e-afe5-4263555d5b7a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.617006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:36.617501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:36.653210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.654020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.654360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.654644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.655413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] RBAC: Adding query filter params (name=image) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:36.655725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.657139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.663013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 533/1050] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:36.668804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.696601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 518/1051] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:36.720890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.780349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.782231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28045c-1805-41ba-8b99-b576f5243528 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.817310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.817993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.818367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.818663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.819291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] RBAC: Adding query filter params (type=image) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:36.819602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.820985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:36.826335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 534/1052] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:36.832250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.839562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e376a4a-c55d-4816-8ed9-99e3bfc4ba3e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.845538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:36.875739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.876550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.876831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.877103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.888813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:36.897057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:36.898013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:36.898872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 519/1053] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:36.904493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:36.949288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99656475-9874-4887-8469-eca71d07fe4e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:36.985595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:36.986554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:36.986918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:36.987559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:36.987969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:36.989534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.001379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:37.009503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd7f94-b36f-4e86-b419-9bf647752bff None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:37.019374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 535/1054] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:03:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:37.067275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:37.067976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:37.068262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:37.068539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:37.069739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:37.069997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:37.071457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.081670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 520/1055] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:37.088054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:37.161254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:37.202442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22059a60-c980-4a53-afd6-c7f2ddc96591 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:37.240782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:37.241683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:37.241963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:37.242240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] PATH_INFO: `/v3/users/6d6742c0012d470d8e911148a7d30945` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:37.256657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] RBAC: Authorizing `identity:get_user(user_id=6d6742c0012d470d8e911148a7d30945)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:37.258124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.260919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 536/1056] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:37 2022] GET /identity/v3/users/6d6742c0012d470d8e911148a7d30945 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:37.267522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:37.374753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ace0-830b-417b-8787-f209dc7d9e0a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:37.409279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:37.409837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:37.410124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:37.410402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:37.432428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:37.479728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:37.481195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.597287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 521/1057] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:37.599051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:37.706889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4e365-f2b9-45e2-9376-df5a55689f1c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:37.732274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:37.744728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:37.745694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:37.746046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:37.746390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:37.747688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:37.749227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e98dc06-5dd2-4a9f-8674-cc60aab6ec68 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.751182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 537/1058] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:37 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:37.757363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:37.811222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:37.858937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45681dc5-f8df-4783-8da4-6f7b0d400a0b None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:37.893436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:37.894217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:37.894599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:37.894973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:37.895940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=449a95490bf044f6bd1e84f990c5ce84, user.id=6d6742c0012d470d8e911148a7d30945) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:37.896384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:37.897926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9261-149e-401e-b6a0-d5dd961c5cbb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:37.903818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 522/1059] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:37 2022] GET /identity/v3/role_assignments?user.id=6d6742c0012d470d8e911148a7d30945&scope.project.id=449a95490bf044f6bd1e84f990c5ce84&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:38.654659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761269d-2a9f-481b-9152-3d55af59d195 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:38.655371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761269d-2a9f-481b-9152-3d55af59d195 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:38.655371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761269d-2a9f-481b-9152-3d55af59d195 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:38.656564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 538/1060] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:38.662219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:38.662628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:38.662768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:38.680258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:38.681443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:38.751221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 523/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:38.773966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761269d-2a9f-481b-9152-3d55af59d195 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:38.879547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761269d-2a9f-481b-9152-3d55af59d195 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:38.915186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:38.916015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:38.916295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:38.916567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:38.927440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-cbb3b1ba-8e3f-495c-bc56-10e6a8d5f553 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:38.935482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:38.936449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:38.937731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 539/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:38.943368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:38.993204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.049871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a0625-ba63-4fe7-9248-e21aeb5b4248 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.085551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.086311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.087393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.087708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.088022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96743e-7097-46c5-95b6-cb659276562b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.088311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96743e-7097-46c5-95b6-cb659276562b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.088470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:39.088565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96743e-7097-46c5-95b6-cb659276562b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.088799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.090071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 540/1063] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:39.090280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.094783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.095067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.095370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.104526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 524/1064] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:39.105402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.113361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:39.114980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:39.164177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.184957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 541/1065] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:39.207778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.216474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96581f80-fccc-45be-b495-cb9d1cb681e2 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.227912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.256319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.257128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.257412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.257689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.269106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.270571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:39.287931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:39.289045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:39.289954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:39.289954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:39.289954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:39.289954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:39.289954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 525/1066] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:39.294869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.315532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57100f36-d84b-4239-95d7-fa3892ddcd5c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.350384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.351055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.351387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.351720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.401905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8cec-a8e9-4722-afcd-abfd88b20717 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.402869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.422259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.423675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.438134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.438814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.439116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.439637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.440213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:39.440583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.442007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.454397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 526/1067] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:39.460061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.481859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.535270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 542/1068] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 04:03:39.537668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.575241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b286fe22-cdb4-4d4c-8e4f-c8ddc369fb82 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.611567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.612403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.612690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.612967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.620904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.622364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.631261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:39.639722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:39.640698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:39.641657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 527/1069] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:39.647046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.652104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc7bf5-bf12-4add-978b-074841461cb0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.686838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.688414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.688878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.689342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.700643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.702329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:39.720255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:39.721377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:39.722295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:39.722295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:39.722295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:39.722295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:39.722295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 543/1070] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:39.727384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.735587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.758311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6af29-8be4-452e-83ed-8d9bd40cb651 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.797861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.798545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.798830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.799146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.800571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:39.800887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.802331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.820251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 528/1071] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:39.825770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:39.842623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7192a9-2e32-4f92-9076-076cfbf4c382 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.881035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.881743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.882028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.882415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.883276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:39.883585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.884963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0179ceec-c0ce-401f-ba67-21558ed57d6f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:39.886792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.898604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 544/1072] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:39.917054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958a43-037b-494c-8d5a-121d7937fd45 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.917378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958a43-037b-494c-8d5a-121d7937fd45 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.917626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89958a43-037b-494c-8d5a-121d7937fd45 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.919069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 545/1073] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:03:39.925515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.925837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.926080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.934182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0851b-6309-40b2-89ae-0c21fc98fd94 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:39.943540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:39.944677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:39.974811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:39.975820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:39.976106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:39.976369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:39.985693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:39.990384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:39.992282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:40.013124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:40.014189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:40.015449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:40.015449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:40.015449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:40.015449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:40.015449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 529/1074] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:40.020575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.022983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 546/1075] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:40.046027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.143353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb95b-5f35-476e-9ca6-9a0d145043d1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.160599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580e318-a996-4161-b363-6550994cd9a5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.182060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.182739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.183027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.183349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.184080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:40.184392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.185784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.188954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:40.197654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 530/1076] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:40.200291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.201151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.201496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.201830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.203403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.212015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:40.219595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:40.220645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:40.221584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 547/1077] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:40.228329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.253631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:40.303698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c304c-dcae-4328-afe2-6198a8741fb4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.327462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a8f03-8b97-40a1-8aae-cd07d3981683 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.337503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.338618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.338896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.339185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.345622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.347662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.360659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.361345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.361680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.361971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:40.362113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:40.362957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:40.364083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:40.364510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:40.364510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.364646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 531/1078] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:40.364714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.369069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.373039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 548/1079] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:40.378485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.423951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:40.469723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77333b02-4ce1-43d8-9d88-559607399f72 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.477558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac0a84d-881f-4646-9797-16a72c7071fb None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.481035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:40.506076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.506743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.507027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.507334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.508117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:40.508428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.509874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.510715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.511641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.511922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.512202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] PATH_INFO: `/v3/users/6d6742c0012d470d8e911148a7d30945` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.513408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] RBAC: Authorizing `identity:get_user(user_id=6d6742c0012d470d8e911148a7d30945)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.514828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.516507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 549/1080] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/users/6d6742c0012d470d8e911148a7d30945 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:40.521906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 532/1081] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:40.522727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.527305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.623488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f77632-e7c1-41fa-8e44-d0722f83127a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.625563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4e7db-65f9-4d28-ba92-490981333c2e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.637303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:40.657742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.658707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.658841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.659431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.659689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.659797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.659976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.660252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/b2f31e54cd054b9aa89d6df02c6b7ec4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.660867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.662312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.662457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=67b4385cb5ed42d1b9488d409abd61b1, role_id=b2f31e54cd054b9aa89d6df02c6b7ec4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.663920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b654b65-48dc-464d-a39f-0d2151eae3bf None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.664008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 550/1082] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:40 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:40.669893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:40.679303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 533/1083] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:40 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/67b4385cb5ed42d1b9488d409abd61b1/roles/b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:03:40.791669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb3c3e-aa0e-477b-b052-41b56eb9b9b1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:40.833067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:40.833866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:40.834149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:40.834437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84/users/6d6742c0012d470d8e911148a7d30945/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:40.836736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a95490bf044f6bd1e84f990c5ce84, user_id=6d6742c0012d470d8e911148a7d30945, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:40.838177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abfa21c-bc22-4d84-b313-d98a65e3d337 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:40.853233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 551/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:40 2022] PUT /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84/users/6d6742c0012d470d8e911148a7d30945/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:03:42.693260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a70315-2ff0-42a3-a61f-e45cf42b6d9b None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:42.693807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a70315-2ff0-42a3-a61f-e45cf42b6d9b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:42.693807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a70315-2ff0-42a3-a61f-e45cf42b6d9b None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:42.695188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 534/1085] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:42.700614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468cf66-c5a3-43a0-9db9-bb49d3342a02 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:42.701063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468cf66-c5a3-43a0-9db9-bb49d3342a02 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:42.701246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468cf66-c5a3-43a0-9db9-bb49d3342a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:42.718565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9468cf66-c5a3-43a0-9db9-bb49d3342a02 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:42.719789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9468cf66-c5a3-43a0-9db9-bb49d3342a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:42.758904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:42.795330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 552/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:03:42.819676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a70315-2ff0-42a3-a61f-e45cf42b6d9b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:42.843313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0699dd-8c1c-47e3-b4c6-816747c89cd5 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:42.843604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0699dd-8c1c-47e3-b4c6-816747c89cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:42.843932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0699dd-8c1c-47e3-b4c6-816747c89cd5 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:42.845309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 553/1087] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:42.850414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:42.850704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:42.850985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:42.851665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:42.868948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:42.870147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:42.926740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a70315-2ff0-42a3-a61f-e45cf42b6d9b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:42.938089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 554/1088] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:42.959404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:42.960231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:42.960514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:42.960791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:42.962227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:42.971378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:42.979534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:42.980487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:42.981332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 535/1089] 10.222.0.13 () {56 vars in 1120 bytes} [Thu Jun 30 04:03:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:42.986722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.064059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62023cda-f987-49ac-8f6d-2b4eef03573f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.065749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.078253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.088495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2103b-9424-413a-91fe-780dae393c83 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.099254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.100063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.100351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.100621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.111269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:03:43.118966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:43.120000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:43.120895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 555/1090] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:03:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:43.124933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.125672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.126016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.126311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.126448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.127298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:43.127608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.128953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.141828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 536/1091] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:03:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:43.142781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.225804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41022c34-8a3e-49db-8af1-747c13a03471 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.237964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c837cb-026f-4bc2-a2f5-85922a8c59bc None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.259861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.260565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.260852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.261136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.261910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:43.262238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.263859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.273648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 556/1092] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:43.275573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.276604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.276926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.277279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.280075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.288874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.290339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.306708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:43.308067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:43.309035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:43.310074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:43.310074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:43.310074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:43.310074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:43.310074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 537/1093] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:43.315429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.318925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.381316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b014fe3-0dd8-431d-9918-987bdec64553 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.414502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de0d2-2acd-4be8-a17f-fd856796c5f6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.415124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.415981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.416298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.416576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] PATH_INFO: `/v3/users/6d6742c0012d470d8e911148a7d30945` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.417768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] RBAC: Authorizing `identity:get_user(user_id=6d6742c0012d470d8e911148a7d30945)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.419247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.420841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 557/1094] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/users/6d6742c0012d470d8e911148a7d30945 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:43.427272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.448294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.448966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.449251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.449530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.450259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:43.450568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.451978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.463671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 538/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:43.469414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.519387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.528963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd02e90-57bb-4b2d-8631-601e408d5061 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.532212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.562697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.563866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.564155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.564435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.573562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80235bae-3feb-4c3a-be06-cb1c9b185fff None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.610133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.610952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.611281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.611904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.619159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.620615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.631745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.633128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:03:43.635902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:43.637014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:03:43.637978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 539/1096] 10.222.0.13 () {56 vars in 1093 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:03:43.643053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.724545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.734760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 558/1097] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 04:03:43.740793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.741874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bc20d3-ebb3-4522-9d7f-9dfbc25213ce None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.766412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.778402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.779077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.779387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.779662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.780498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:43.780803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.782244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.801096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 540/1098] 10.222.0.13 () {56 vars in 1133 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:43.806854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.845551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d6ebb-5602-4b88-9338-579a3bc4ff1b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.880882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.881824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.882054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.882332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.883568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:43.885018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:43.886747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 559/1099] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:03:43.892841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:43.909095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068fd63-2887-4b5e-b9c7-9ae3f3603e9d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:43.943573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:43.944230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:43.944510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:43.944783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:43.951720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.992593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:43.992799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f550e8b9-fc9b-4c65-8e97-2562bbb05da4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.011436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.012904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.028487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.029412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.029819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.030246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.031472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=449a95490bf044f6bd1e84f990c5ce84, user.id=6d6742c0012d470d8e911148a7d30945) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:44.031841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.033268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.040007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 560/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/role_assignments?user.id=6d6742c0012d470d8e911148a7d30945&scope.project.id=449a95490bf044f6bd1e84f990c5ce84&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:44.121121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 541/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 04:03:44.127400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:44.239821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428fc00-8d01-4526-be21-ff54af53cc04 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.262703 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:44.273339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.274150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.274435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.274711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.285230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.286692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:03:44.303736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:44.304658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:44.305559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:03:44.305559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:03:44.305559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:03:44.305559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:44.305559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 561/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:44.310789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:44.395616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:44.414755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383637-b7f7-4ebf-975d-21d6438bf0fa None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.450295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.450958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.451272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.451545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.452269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:44.452579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.453947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.465960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 542/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:44.471700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:44.573472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23329e-fbdc-4247-ae6b-a4cae43b4cb8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.610552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.611391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.611651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.611924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.618731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.620224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:03:44.634408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:03:44.635332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:03:44.636290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 562/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:03:44.641837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:44.748570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1a4d7-7469-4cd0-8d18-c1436d5abeda None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.783667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.784374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.784653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.784928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.785725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:44.786038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.787520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad34f2-e56c-4dc5-9109-5cda22b5d8c2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.800833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 543/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:44.806971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:44.842770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:44.909520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346e951-8adb-4e73-b9fa-24b7fc1ad984 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:44.944629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:44.945281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:44.945566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:44.945845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:44.946674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] RBAC: Adding query filter params (role.id=b2f31e54cd054b9aa89d6df02c6b7ec4, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=67b4385cb5ed42d1b9488d409abd61b1) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:44.946990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:44.948453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aae4fe-6ba1-4164-8380-36a32ee662c8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:44.954582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 563/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:44 2022] GET /identity/v3/role_assignments?user.id=67b4385cb5ed42d1b9488d409abd61b1&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=b2f31e54cd054b9aa89d6df02c6b7ec4 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:46.114744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb0b4-fe3f-453d-8108-ee8b24530c12 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:46.115662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb0b4-fe3f-453d-8108-ee8b24530c12 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:46.115662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb0b4-fe3f-453d-8108-ee8b24530c12 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:46.117840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 544/1107] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:46.124086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:46.124752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:46.124960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:46.143998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:46.145184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:46.216080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 564/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:46.242122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0eb0b4-fe3f-453d-8108-ee8b24530c12 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:46.246813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:46.353648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0eb0b4-fe3f-453d-8108-ee8b24530c12 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:46.389035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:46.389756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:46.390037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:46.390312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:46.391613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:46.393536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d23d3f-0072-47f6-8740-d868a0346b36 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:46.395152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 545/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:46.401659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:46.435361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:46.504495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1039498-53b6-4a82-a2ea-6f48aec6fa89 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:46.542811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:46.543505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:46.543789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:46.544068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:46.544778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:46.546219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56460f-d805-4554-8b5b-3ae190aa2f5f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:46.571065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 565/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:03:46 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:03:48.612824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7187-8449-4344-9dba-b40ef9ac23fa None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:48.613331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7187-8449-4344-9dba-b40ef9ac23fa None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:48.613331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d7187-8449-4344-9dba-b40ef9ac23fa None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:48.614763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 546/1111] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:48.620392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:48.620808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:48.620915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:48.639371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:48.640579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:48.711731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 566/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:48.745264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7187-8449-4344-9dba-b40ef9ac23fa None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:48.834035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:48.868939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d7187-8449-4344-9dba-b40ef9ac23fa None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:48.903914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:48.904768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:48.905048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:48.905321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] PATH_INFO: `/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:48.906928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] RBAC: Authorizing `identity:get_role(role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:48.908702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:48.909776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 547/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:48 2022] GET /identity/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:48.919286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:48.938982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:49.025428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91203b5e-ba9d-40a6-ad15-0313253fa7cf None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:49.061384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:49.062247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:49.062530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:49.062804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] PATH_INFO: `/v3/users/12d8552ac6e2407b95c3cf611aa14b26` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:49.077558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] RBAC: Authorizing `identity:get_user(user_id=12d8552ac6e2407b95c3cf611aa14b26)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:49.079018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:49.081184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 567/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:48 2022] GET /identity/v3/users/12d8552ac6e2407b95c3cf611aa14b26 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:49.090119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:49.196287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669c5e1-e079-4a61-bdf5-9ab70b8faa83 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:49.232150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:49.232806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:49.233090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:49.233375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:49.241508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:49.304615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:49.306073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:49.408750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 548/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:49.414996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:49.494581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:49.524893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ffc12-7a36-4271-a396-a130e626ab32 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:49.560729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:49.561585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:49.561870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:49.562145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:49.563427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:49.564878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8c73a-ab37-4e7c-821e-41f75ab944e5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:49.566553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 568/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:49 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:49.575411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:49.618237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:49.689776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628647-e2d5-404b-bcfb-ee88bf86fe0a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:49.733131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:49.734100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:49.734455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:49.734829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:49.736170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=449a95490bf044f6bd1e84f990c5ce84, user.id=12d8552ac6e2407b95c3cf611aa14b26) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:49.736547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:49.737998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754adb25-009c-408a-b4af-5af66a9144b1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:49.743726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 549/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:49 2022] GET /identity/v3/role_assignments?user.id=12d8552ac6e2407b95c3cf611aa14b26&scope.project.id=449a95490bf044f6bd1e84f990c5ce84&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:51.771558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8fb5d-fca3-4421-b5a3-eafb7ff2fdac None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:51.772489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8fb5d-fca3-4421-b5a3-eafb7ff2fdac None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:51.772489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c8fb5d-fca3-4421-b5a3-eafb7ff2fdac None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:51.774626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 569/1118] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:51.780389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:51.780829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:51.780932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:51.799943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:51.801144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:51.871010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 550/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:51.895157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8fb5d-fca3-4421-b5a3-eafb7ff2fdac None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:52.023283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c8fb5d-fca3-4421-b5a3-eafb7ff2fdac None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:52.059218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:52.060068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:52.060351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:52.060627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] PATH_INFO: `/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:52.062194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] RBAC: Authorizing `identity:get_role(role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:52.063614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:52.065558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 570/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:51 2022] GET /identity/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:52.072636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:52.111702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:52.183800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbf3e3-1809-4695-b25c-f92f00262b28 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:52.230723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:52.231640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:52.231933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:52.232209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] PATH_INFO: `/v3/users/12d8552ac6e2407b95c3cf611aa14b26` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:52.233463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] RBAC: Authorizing `identity:get_user(user_id=12d8552ac6e2407b95c3cf611aa14b26)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:52.234916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:52.236633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 551/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:52 2022] GET /identity/v3/users/12d8552ac6e2407b95c3cf611aa14b26 => generated 286 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:52.242892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:52.256435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:52.354975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adebadb-1d2e-432b-bb2e-5490330c8420 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:52.395967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:52.396813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:52.397089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:52.397355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:52.398546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:52.400018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aea92b-2fb7-4813-8fef-217cfb994fc9 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:52.401722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 571/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:52 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:52.410913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:52.517843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:52.521237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d7983-8c9e-4004-ab07-ec0bbec04762 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:52.558227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:52.559141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:52.559419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:52.559689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84/users/12d8552ac6e2407b95c3cf611aa14b26/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:52.562181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] RBAC: Authorizing `identity:create_grant(project_id=449a95490bf044f6bd1e84f990c5ce84, user_id=12d8552ac6e2407b95c3cf611aa14b26, role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:52.563734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15417ac2-48f4-4652-9206-3a26b1405343 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:52.579889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 552/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:03:52 2022] PUT /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84/users/12d8552ac6e2407b95c3cf611aa14b26/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:03:54.580870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063a65-f3d9-4cc5-b849-a507f87aad05 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:54.581581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063a65-f3d9-4cc5-b849-a507f87aad05 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:54.581831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80063a65-f3d9-4cc5-b849-a507f87aad05 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:54.584016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 572/1124] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:54.592711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:54.593145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:54.593322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:54.612616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:54.613828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:54.687924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 553/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:54.712467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063a65-f3d9-4cc5-b849-a507f87aad05 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:54.803796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:54.831754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80063a65-f3d9-4cc5-b849-a507f87aad05 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:54.866543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:54.867433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:54.867715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:54.867999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] PATH_INFO: `/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:54.870214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] RBAC: Authorizing `identity:get_role(role_id=1b1b49ed2d054b75b922d0819ed7eb70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:54.870917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:54.872327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 573/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:54 2022] GET /identity/v3/roles/1b1b49ed2d054b75b922d0819ed7eb70 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:54.878502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:54.973397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:54.984906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93400cd-37d0-4ffd-b081-4c972dac2f7c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:55.019804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:55.020682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:55.020962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:55.021242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] PATH_INFO: `/v3/users/12d8552ac6e2407b95c3cf611aa14b26` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:55.022551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] RBAC: Authorizing `identity:get_user(user_id=12d8552ac6e2407b95c3cf611aa14b26)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:55.024020 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:55.025649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 554/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:03:54 2022] GET /identity/v3/users/12d8552ac6e2407b95c3cf611aa14b26 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:55.032318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:55.135709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e69b3-4a42-486c-a824-3c61d4d45365 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:55.172868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:55.173508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:55.173823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:55.174886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:55.190427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:55.245220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:55.246655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:55.349493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 574/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:03:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:03:55.355863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:55.467480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f068ef-1147-47a5-bef1-221cbc4e62a1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:55.521336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:55.522205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:55.522489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:55.522770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] PATH_INFO: `/v3/projects/449a95490bf044f6bd1e84f990c5ce84` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:55.524123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] RBAC: Authorizing `identity:get_project(project_id=449a95490bf044f6bd1e84f990c5ce84)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:55.525575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5771b-0254-48ca-b654-beaf4de35a18 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:55.527333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 555/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:03:55 2022] GET /identity/v3/projects/449a95490bf044f6bd1e84f990c5ce84 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:55.536417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:55.588785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:55.643482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718380f2-dbc4-48c7-8a82-81e40da40bbc None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:55.679563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:55.680217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:55.680498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:55.680775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:55.681599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] RBAC: Adding query filter params (role.id=1b1b49ed2d054b75b922d0819ed7eb70, scope.project.id=449a95490bf044f6bd1e84f990c5ce84, user.id=12d8552ac6e2407b95c3cf611aa14b26) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:03:55.681911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:55.683362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ab3ca-9ced-4170-b642-c0ec656f74b5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:55.689245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 575/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:03:55 2022] GET /identity/v3/role_assignments?user.id=12d8552ac6e2407b95c3cf611aa14b26&scope.project.id=449a95490bf044f6bd1e84f990c5ce84&role.id=1b1b49ed2d054b75b922d0819ed7eb70 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:57.713118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd9ae-174f-4bae-8daa-522382e3b447 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:57.713702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd9ae-174f-4bae-8daa-522382e3b447 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:57.713702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd9ae-174f-4bae-8daa-522382e3b447 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:57.715037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 556/1131] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:03:57.721374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:57.722030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:57.722201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:57.744984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:03:57.746190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:03:57.815634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 576/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:03:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:03:57.839627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dd9ae-174f-4bae-8daa-522382e3b447 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:57.870455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:57.957465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231dd9ae-174f-4bae-8daa-522382e3b447 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:57.995303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:57.996098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:57.996379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:57.996628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:57.997954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:57.999424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b7bc1-0da3-4908-b7db-5ea12ff0f3cf None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:58.001059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 557/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:03:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:03:58.010002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:03:58.059654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:03:58.122553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9218e43-027e-4d85-af4b-efa16091fee4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:03:58.158369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:03:58.159032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:03:58.159354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:03:58.159634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:03:58.160431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:03:58.161886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d31f9-7fb7-4132-9e1a-3194b0b21597 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:03:58.189735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 577/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:03:58 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:04:00.224704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3ecde-6351-4285-889c-b46292719d9d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:00.225563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3ecde-6351-4285-889c-b46292719d9d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:00.225563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3ecde-6351-4285-889c-b46292719d9d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:00.227918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 558/1135] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:00.236746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:00.237425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:00.237687 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:00.257296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:00.258488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:00.334286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 578/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:00.359459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3ecde-6351-4285-889c-b46292719d9d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:00.468481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:00.477368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3ecde-6351-4285-889c-b46292719d9d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:00.512064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:00.512856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:00.513146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:00.513419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:00.514652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:00.516130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c327a9-c9e3-4b4d-a7ea-c32d6d3f3c18 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:00.517753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 559/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:00.523704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:00.550134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:00.632528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec10a0-2203-4a2e-b1cc-4f4d81fe2fd4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:00.667561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:00.668234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:00.668522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:00.668802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:00.669515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:00.670962 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a7a4ca-221e-463f-8d4e-9df444145a49 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:00.694259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 579/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:04:00 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:04:02.731150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234bf5-2beb-4ec5-9c8c-5f077045051e None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:02.731987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234bf5-2beb-4ec5-9c8c-5f077045051e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:02.732168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c234bf5-2beb-4ec5-9c8c-5f077045051e None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:02.734292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 560/1139] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:02.741186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:02.741922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:02.742045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:02.765326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:02.766520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:02.838796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 580/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:02.866221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c234bf5-2beb-4ec5-9c8c-5f077045051e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:02.985727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c234bf5-2beb-4ec5-9c8c-5f077045051e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:03.022088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:03.022889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:03.023206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:03.023486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:03.034323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:03.042063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:03.043148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:03.044089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 561/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:03.052484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:03.068258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:03.176450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6456f-cb57-4d69-8cf9-499d48fefbab None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:03.216857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:03.217529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:03.217820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:03.218097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:03.218849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:03.219197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:03.220592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:03.229643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 581/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:03.235895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:03.264154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:03.343803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84668696-0cfd-4135-82eb-7599d572747a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:03.377962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:03.378876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:03.379203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:03.379462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] PATH_INFO: `/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:03.400987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] RBAC: Authorizing `identity:get_user(user_id=8196bbe3810a4a02b46cb5a3a8b4c8a5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:03.403333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:03.405942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 562/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:03 2022] GET /identity/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:03.414072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:03.497505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:03.528189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a89338-e945-494a-b2cc-c1cb424cb109 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:03.564280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:03.564912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:03.565195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:03.565470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:03.646416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:03.647897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:03.717473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:03.757704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 582/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:04:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:04:03.764031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:03.865615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:03.885569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468d436-d9d9-4954-b6b0-77636441193f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:03.920325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:03.921196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:03.921482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:03.921760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] PATH_INFO: `/v3/projects/50e888e85d2a4163a8d2a0985d174fa6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:03.922983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] RBAC: Authorizing `identity:get_project(project_id=50e888e85d2a4163a8d2a0985d174fa6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:03.924825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae72ec-df13-48d4-8202-83e4c374e490 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:03.926618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 563/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:03 2022] GET /identity/v3/projects/50e888e85d2a4163a8d2a0985d174fa6 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:03.933573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:04.043813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be40d8-732a-4075-b27c-6c88fbe1d14d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:04.080499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:04.081153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:04.081438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:04.081835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:04.082881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=50e888e85d2a4163a8d2a0985d174fa6, user.id=8196bbe3810a4a02b46cb5a3a8b4c8a5) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:04.083225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:04.084665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa08124-d9c8-441e-a5e2-8228da77c32a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:04.090061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 583/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:04:03 2022] GET /identity/v3/role_assignments?user.id=8196bbe3810a4a02b46cb5a3a8b4c8a5&scope.project.id=50e888e85d2a4163a8d2a0985d174fa6&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:06.103335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b218e00-641b-4aa2-8cc4-81f9ac7ceab4 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.104292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b218e00-641b-4aa2-8cc4-81f9ac7ceab4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.104292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b218e00-641b-4aa2-8cc4-81f9ac7ceab4 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.106476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 564/1147] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:06.113522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.113978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.114084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.132066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:06.133286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:06.165650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:06.211892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 584/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:06.235678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b218e00-641b-4aa2-8cc4-81f9ac7ceab4 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:06.346855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b218e00-641b-4aa2-8cc4-81f9ac7ceab4 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:06.381338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:06.382064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.382305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.382604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.393318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:06.401056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:06.402143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:06.403168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 565/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:06.409186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:06.526309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc33ba-5242-4ae4-88c5-800feed5e095 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:06.563829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:06.564510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.564794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.565042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.565797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:06.566185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:06.567709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:06.576278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 585/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:06.583846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:06.597089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:06.695474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2ab3-5522-4e9b-bd89-f0efc1952214 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:06.731504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:06.732361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.732635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.732905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] PATH_INFO: `/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.734131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=8196bbe3810a4a02b46cb5a3a8b4c8a5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:06.735600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:06.737211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 566/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:06 2022] GET /identity/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:06.743943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:06.772620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:06.861642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e3a54-3457-4c76-9ebd-8eaf46219823 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:06.901575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:06.902430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:06.902712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:06.902988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] PATH_INFO: `/v3/projects/50e888e85d2a4163a8d2a0985d174fa6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:06.904236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] RBAC: Authorizing `identity:get_project(project_id=50e888e85d2a4163a8d2a0985d174fa6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:06.905688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaae044-ce4c-4b96-92a7-24d0401f9763 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:06.907396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 586/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:06 2022] GET /identity/v3/projects/50e888e85d2a4163a8d2a0985d174fa6 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:04:06.914290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:07.001422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:07.037483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e80e9d-8004-42a9-a948-cfee12111bd4 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:07.074268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:07.075145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:07.075435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:07.075719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] PATH_INFO: `/v3/projects/50e888e85d2a4163a8d2a0985d174fa6/users/8196bbe3810a4a02b46cb5a3a8b4c8a5/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:07.077909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=50e888e85d2a4163a8d2a0985d174fa6, user_id=8196bbe3810a4a02b46cb5a3a8b4c8a5, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:07.079385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e24dc8-1a3e-4111-919c-b5724f3ec0f5 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:07.095292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 567/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:04:06 2022] PUT /identity/v3/projects/50e888e85d2a4163a8d2a0985d174fa6/users/8196bbe3810a4a02b46cb5a3a8b4c8a5/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:04:09.088691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aeafd-5435-4a20-a812-7b69d699dd40 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.089271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aeafd-5435-4a20-a812-7b69d699dd40 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.089271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aeafd-5435-4a20-a812-7b69d699dd40 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.090583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 587/1154] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:09.097191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.097852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.098143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.121762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:09.122959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:09.201175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 568/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:09.229481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aeafd-5435-4a20-a812-7b69d699dd40 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:09.340452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:09.349106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aeafd-5435-4a20-a812-7b69d699dd40 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:09.386912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:09.387741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.388030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.388307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.401561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:09.412987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:09.414031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:04:09.416355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 588/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:09.421063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:09.494828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:09.539947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d88a2-151d-4131-a83a-fda98765e5f4 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:09.579110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:09.579791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.580079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.580352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.581095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:09.581401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:09.582892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:09.591365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 569/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:09.597518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:09.707069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774915cc-84c6-4f56-87e3-5b9c49013e1f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:09.752100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:09.752985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.753277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.753553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] PATH_INFO: `/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.754959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=8196bbe3810a4a02b46cb5a3a8b4c8a5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:09.756442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:09.758056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 589/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:09 2022] GET /identity/v3/users/8196bbe3810a4a02b46cb5a3a8b4c8a5 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:09.763999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:09.876596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f3834-b0c9-4a94-80d6-3ef2d0a7e818 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:09.902746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:09.912528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:09.913180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:09.913447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:09.913729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:09.982886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:09.984383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:10.087834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 570/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:04:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:04:10.094252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:10.123728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:10.205078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a41cb8-f0f7-4967-b47c-036c49692cc6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:10.240194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:10.241066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:10.241345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:10.241616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] PATH_INFO: `/v3/projects/50e888e85d2a4163a8d2a0985d174fa6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:10.242840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] RBAC: Authorizing `identity:get_project(project_id=50e888e85d2a4163a8d2a0985d174fa6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:10.244321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e2d62-7ebe-47e8-8a2a-dbd0e7184ed7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:10.246013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 590/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:10 2022] GET /identity/v3/projects/50e888e85d2a4163a8d2a0985d174fa6 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:10.253074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:10.281411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:10.358113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d34b19-26ee-4238-93b1-b1497a0f9261 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:10.393389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:10.394036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:10.394317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:10.394587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:10.395970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=50e888e85d2a4163a8d2a0985d174fa6, user.id=8196bbe3810a4a02b46cb5a3a8b4c8a5) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:10.396328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:10.397757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fb80d-3cd0-4b30-a8c9-a6742acec463 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:10.403878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 571/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:04:10 2022] GET /identity/v3/role_assignments?user.id=8196bbe3810a4a02b46cb5a3a8b4c8a5&scope.project.id=50e888e85d2a4163a8d2a0985d174fa6&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:12.452578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469dce-8ba6-4159-89af-173c037c7a63 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:12.453473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469dce-8ba6-4159-89af-173c037c7a63 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:12.453473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2469dce-8ba6-4159-89af-173c037c7a63 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:12.455701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 591/1162] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:12.461081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:12.461521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:12.461631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:12.482427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:12.484372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:12.568345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 572/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:12.591691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2469dce-8ba6-4159-89af-173c037c7a63 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:12.695269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:12.708443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2469dce-8ba6-4159-89af-173c037c7a63 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:12.742615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:12.743305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:12.743590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:12.743874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:12.810881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:12.812397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:12.902786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:12.914532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 592/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:04:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:04:12.921625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:12.949614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:13.029262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880456c-0344-4e3f-870c-dfcb818af0cd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:13.064937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:13.065737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:13.066017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:13.066292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:13.073879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:13.075367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-1ee93daf-7c72-4bae-850f-d9a54e8cc67a None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:13.089859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:13.091012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:13.091914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 573/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:04:12 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:13.097495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:13.211599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b286f2-b885-4be5-ac65-61563f9e6022 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:13.247179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:13.247847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:13.248134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:13.248412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:13.249141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:13.249454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:13.250836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7715b3-b70d-46e2-8d74-f94e25414c8b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:13.258315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 593/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Thu Jun 30 04:04:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:15.245162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac375f5-16ad-4eae-9361-7cb833315b24 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:15.246010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac375f5-16ad-4eae-9361-7cb833315b24 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:15.246473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac375f5-16ad-4eae-9361-7cb833315b24 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:15.249970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 574/1167] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:15.255705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc94d-7bea-4789-b1f5-bfc7231bdd2c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:15.256145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc94d-7bea-4789-b1f5-bfc7231bdd2c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:15.256233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dc94d-7bea-4789-b1f5-bfc7231bdd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:15.278518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a16dc94d-7bea-4789-b1f5-bfc7231bdd2c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:15.280480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a16dc94d-7bea-4789-b1f5-bfc7231bdd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:15.355621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:15.374922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 594/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:15.399628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac375f5-16ad-4eae-9361-7cb833315b24 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:15.495058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:15.514990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac375f5-16ad-4eae-9361-7cb833315b24 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:15.550009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:15.550663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:15.551010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:15.551339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:15.552081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:15.553457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81103d8a-f75e-40ca-ae76-f21a29ca6239 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:15.570781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 575/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Thu Jun 30 04:04:15 2022] POST /identity/v3/domains => generated 267 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:04:17.617881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ae194-efbf-4305-8351-378bfbc28592 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:17.618708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ae194-efbf-4305-8351-378bfbc28592 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:17.618855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ae194-efbf-4305-8351-378bfbc28592 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:17.621073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 595/1170] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:17.627577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:17.628031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:17.628159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:17.648250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:17.649443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:17.727243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 576/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:17.763085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ae194-efbf-4305-8351-378bfbc28592 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:17.886335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ae194-efbf-4305-8351-378bfbc28592 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:17.921792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:17.922653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:17.922934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:17.923245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] PATH_INFO: `/v3/domains/fa083a5b0a6d432d973dcdf22db72b78` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:17.935531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fa083a5b0a6d432d973dcdf22db72b78)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:17.936993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c632c-cca5-4ec4-9332-f654e2e35d43 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:17.938672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 596/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jun 30 04:04:17 2022] GET /identity/v3/domains/fa083a5b0a6d432d973dcdf22db72b78 => generated 267 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:17.945683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:18.029132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:18.060296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633a1c0-7824-47dc-9839-2ca53307a42f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:18.095858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:18.096494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:18.096782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:18.097092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:18.097804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:18.099449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392eee4d-3d9a-46c1-96a1-eab09d6d00c2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:18.127129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 577/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:04:17 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:04:20.191714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbc74-7c0d-44d2-bf8a-5e41daddb41e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:20.192575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbc74-7c0d-44d2-bf8a-5e41daddb41e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:20.192575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80cbc74-7c0d-44d2-bf8a-5e41daddb41e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:20.194829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 597/1174] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:20.201339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:20.201782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:20.201945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:20.222093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:20.223313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:20.304473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 578/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:20.337581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cbc74-7c0d-44d2-bf8a-5e41daddb41e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:20.397808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:20.455770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80cbc74-7c0d-44d2-bf8a-5e41daddb41e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:20.495407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:20.496708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:20.497034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:20.497415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] PATH_INFO: `/v3/domains/fa083a5b0a6d432d973dcdf22db72b78` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:20.499565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fa083a5b0a6d432d973dcdf22db72b78)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:20.501853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85d073-0b3a-44d1-a971-56cfd51ed078 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:20.504372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 598/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Thu Jun 30 04:04:20 2022] GET /identity/v3/domains/fa083a5b0a6d432d973dcdf22db72b78 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:20.510776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:20.578035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:20.629374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae481c8-df2a-477f-9a46-4268de35b4d8 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:20.663369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:20.664038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:20.664320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:20.664595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:20.665294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:20.666733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6fa54-7a78-455a-bcb5-0dd0ccc66941 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:20.688896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 579/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:04:20 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:04:22.744677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ef7ff-5bb9-4895-8967-1f05b770cda9 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:22.745308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ef7ff-5bb9-4895-8967-1f05b770cda9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:22.745308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ef7ff-5bb9-4895-8967-1f05b770cda9 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:22.746581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 599/1178] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:22.753058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:22.753790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:22.753946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:22.773844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:22.775038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:22.849850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 580/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:22.886344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ef7ff-5bb9-4895-8967-1f05b770cda9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.004507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ef7ff-5bb9-4895-8967-1f05b770cda9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:23.017631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:23.040064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:23.040860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:23.041144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:23.041413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:23.052285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:23.059956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:23.061111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:04:23.062405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 600/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:23.067780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.108897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:23.167455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7161d2-2e68-449c-a6e3-be54b1c6dfbd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:23.203170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:23.203840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:23.204126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:23.204403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:23.205156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:23.205465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:23.206841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:23.215658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 581/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:23.221827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.324798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61092a-45e6-4bca-9fdb-c3b3c3fc175a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:23.359699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:23.360484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:23.360761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:23.361034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] PATH_INFO: `/v3/users/2a5b2d932706402b980f28ac6fe7183f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:23.374800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5b2d932706402b980f28ac6fe7183f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:23.376289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:23.377913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 601/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:23 2022] GET /identity/v3/users/2a5b2d932706402b980f28ac6fe7183f => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:23.383989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.483457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4131-4a86-4c29-a624-1fdc5b2057b1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:23.485087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:23.518485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:23.519134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:23.519452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:23.519727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:23.591896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:23.594164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:23.697697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 582/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:04:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:04:23.707166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.746976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:23.823990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc7f7e-a208-4a20-8e95-7bf7a342bde0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:23.866283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:23.867208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:23.867492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:23.867773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] PATH_INFO: `/v3/projects/72480bdb72a5480b8aa4134a4ea28eff` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:23.869026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] RBAC: Authorizing `identity:get_project(project_id=72480bdb72a5480b8aa4134a4ea28eff)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:23.870483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d926c5c-885b-44d8-b3a1-74286354bdc1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:23.872211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 602/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:23 2022] GET /identity/v3/projects/72480bdb72a5480b8aa4134a4ea28eff => generated 361 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:23.882520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:23.885710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:23.992296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b1dbf-db51-42eb-9b1d-14cb390bebfb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:24.026948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:24.027619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:24.027900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:24.028178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:24.028986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=72480bdb72a5480b8aa4134a4ea28eff, user.id=2a5b2d932706402b980f28ac6fe7183f) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:24.029295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:24.030732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb2a5-0c8a-48b9-b4ca-335120147a18 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:24.036324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 583/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:04:23 2022] GET /identity/v3/role_assignments?user.id=2a5b2d932706402b980f28ac6fe7183f&scope.project.id=72480bdb72a5480b8aa4134a4ea28eff&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:26.027243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505adea-ad7e-4a3a-88a0-40c3b0a8fb7b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.027815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505adea-ad7e-4a3a-88a0-40c3b0a8fb7b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.027815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505adea-ad7e-4a3a-88a0-40c3b0a8fb7b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.029190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 603/1186] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:26.035644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.036453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.036605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.056975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:26.058721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:26.131018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:26.140094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 584/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:26.165533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505adea-ad7e-4a3a-88a0-40c3b0a8fb7b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:26.282208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:26.287370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505adea-ad7e-4a3a-88a0-40c3b0a8fb7b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:26.323085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:26.323920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.324208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.324487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.335063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:26.342756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:26.343862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:04:26.344733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 604/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:26.353268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:26.461143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434a83-6cc5-42c2-b120-891e7134a43b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:26.501420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:26.502093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.502375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.502647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.503432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:26.503742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:26.505200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:26.514149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 585/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:26.521123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:26.633272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a107e08-7d4e-4d0e-aa61-de5f85b17cd5 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:26.671327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:26.672095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.672376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.672651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] PATH_INFO: `/v3/users/2a5b2d932706402b980f28ac6fe7183f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.674243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a5b2d932706402b980f28ac6fe7183f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:26.675727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:26.677566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 605/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:26 2022] GET /identity/v3/users/2a5b2d932706402b980f28ac6fe7183f => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:26.687392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:26.753592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:26.796268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f5883-fd20-468d-9cf6-8cb558f07451 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:26.831513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:26.832307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.832585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.832855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] PATH_INFO: `/v3/projects/72480bdb72a5480b8aa4134a4ea28eff` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.834502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] RBAC: Authorizing `identity:get_project(project_id=72480bdb72a5480b8aa4134a4ea28eff)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:26.835974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62077e-c1fc-4ddd-b15d-81e955b56c44 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:26.837662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 586/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:26 2022] GET /identity/v3/projects/72480bdb72a5480b8aa4134a4ea28eff => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:26.843899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:26.873749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:26.959043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238570e0-277c-45ec-b22d-a3aec97292b6 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:26.992371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:26.993294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:26.993575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:26.993849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] PATH_INFO: `/v3/projects/72480bdb72a5480b8aa4134a4ea28eff/users/2a5b2d932706402b980f28ac6fe7183f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:26.995823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] RBAC: Authorizing `identity:create_grant(project_id=72480bdb72a5480b8aa4134a4ea28eff, user_id=2a5b2d932706402b980f28ac6fe7183f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:26.997264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223e801-1569-4a42-af87-c1d1b7487a82 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:27.012243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 606/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:04:26 2022] PUT /identity/v3/projects/72480bdb72a5480b8aa4134a4ea28eff/users/2a5b2d932706402b980f28ac6fe7183f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:04:29.000444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7ba5-373e-42cb-9070-33776c1d62a8 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.001016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7ba5-373e-42cb-9070-33776c1d62a8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.001016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb7ba5-373e-42cb-9070-33776c1d62a8 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.002392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 587/1193] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:04:29.008970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196152-78cb-4639-9762-72781aee2411 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.009403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196152-78cb-4639-9762-72781aee2411 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.009502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196152-78cb-4639-9762-72781aee2411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.037205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1d196152-78cb-4639-9762-72781aee2411 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:04:29.038391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1d196152-78cb-4639-9762-72781aee2411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:04:29.108923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 607/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:04:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:04:29.134781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb7ba5-373e-42cb-9070-33776c1d62a8 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:29.253022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb7ba5-373e-42cb-9070-33776c1d62a8 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:29.287414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:29.303315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:29.304155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.304441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.304715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.322757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:04:29.334797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:04:29.335951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:04:29.337349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 588/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:04:29.345591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196152-78cb-4639-9762-72781aee2411 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:29.370306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:29.457803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196152-78cb-4639-9762-72781aee2411 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:29.496286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:29.496960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.497245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.497522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.498254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:29.498563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:29.499982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:29.510852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 608/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:29.519416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:29.622662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ca93a-e5dd-4dd4-b270-c02e46640480 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:29.658230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:29.659570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.659938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.660312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] PATH_INFO: `/v3/users/2a5b2d932706402b980f28ac6fe7183f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.662215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] RBAC: Authorizing `identity:get_user(user_id=2a5b2d932706402b980f28ac6fe7183f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:29.663780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:29.665434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 589/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Thu Jun 30 04:04:29 2022] GET /identity/v3/users/2a5b2d932706402b980f28ac6fe7183f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:29.671549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:29.765692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:29.790413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dd0b8-3908-455a-8d62-430bfb69c7e1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:29.830618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:29.831739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:29.832208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:29.832664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:29.903479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:29.904919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:29.987656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:30.023882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 609/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:04:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:04:30.030145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:30.091004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:30.138008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18928f-3413-4816-8e2f-e9afdfad4e2c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:30.173362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:30.174583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:30.174871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:30.175155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] PATH_INFO: `/v3/projects/72480bdb72a5480b8aa4134a4ea28eff` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:30.176474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] RBAC: Authorizing `identity:get_project(project_id=72480bdb72a5480b8aa4134a4ea28eff)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:30.177950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4d449-963d-4d25-8fd2-867af931d72b None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:30.179700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 590/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Thu Jun 30 04:04:30 2022] GET /identity/v3/projects/72480bdb72a5480b8aa4134a4ea28eff => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:30.189461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:04:30.316270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21672ccc-9868-4f4f-ad14-bff08ff0f893 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:04:30.352760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:04:30.353412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:04:30.353694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:04:30.353969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:04:30.354793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=72480bdb72a5480b8aa4134a4ea28eff, user.id=2a5b2d932706402b980f28ac6fe7183f) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:04:30.355147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:04:30.356630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384a465-3777-4b9a-af09-1efc395c6df2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:04:30.363114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 610/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:04:30 2022] GET /identity/v3/role_assignments?user.id=2a5b2d932706402b980f28ac6fe7183f&scope.project.id=72480bdb72a5480b8aa4134a4ea28eff&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:04:40.167991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=39 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:04:40.360111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=79 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:26.104398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0798d09-3553-474d-aa02-ad8286ec18ee None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:26.105071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0798d09-3553-474d-aa02-ad8286ec18ee None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:26.105071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0798d09-3553-474d-aa02-ad8286ec18ee None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:26.106660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 591/1201] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:26.115195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:26.115195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:26.115195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:26.131695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:26.133247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:26.210768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 611/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:26.238356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0798d09-3553-474d-aa02-ad8286ec18ee None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:26.378720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0798d09-3553-474d-aa02-ad8286ec18ee None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:26.427989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:26.428957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:26.429317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:26.429734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:26.442210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:26.443849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:26.462577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:26.463849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:26.464890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:26.464890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:26.464890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:26.464890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:26.466887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 592/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:26.474959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:26.613814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b3001-6472-47d9-b98c-7346161697ba None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:26.653075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:26.659625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:26.660943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:26.661339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:26.661790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:26.662540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:26.662852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:26.664294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00371ee8-0370-44e2-9612-6dc3d4ac3317 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:26.684274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 612/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:26.692425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:26.751898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:26.808021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5528be-1d98-4288-917a-d98e07fd4564 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:26.845258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:26.846050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:26.846269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:26.846594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:26.847365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:26.849011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad6c97-76a1-471b-b0c9-9c147453b009 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:26.872553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 593/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Thu Jun 30 04:05:26 2022] POST /identity/v3/users => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:05:29.009824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461979d5-7ec4-4e46-8711-a41de8fcdca2 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.010410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461979d5-7ec4-4e46-8711-a41de8fcdca2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.010498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461979d5-7ec4-4e46-8711-a41de8fcdca2 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.012017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 613/1206] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:29.018154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.018692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.018812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.043124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:29.043124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:29.117169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 594/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:29.142967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461979d5-7ec4-4e46-8711-a41de8fcdca2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:29.255274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461979d5-7ec4-4e46-8711-a41de8fcdca2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:29.293049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:29.294057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.294379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.294696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.314161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:29.322416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:29.323749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:29.325084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 614/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:05:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:29.329895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:29.380369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:29.459485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60d99-9eb9-4c81-b3da-9139d917967e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:29.501479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:29.502198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.502487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.502768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.503602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:29.503985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:29.505450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:29.515064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 595/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:05:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:29.521326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:29.577134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:29.635951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2931247-8347-414b-9cc1-e4cc1a413059 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:29.673485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:29.674305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.674593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.674909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.687156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:29.688667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:29.708157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:29.709383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:29.710929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:29.710929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:29.710929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:29.710929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:29.710929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 615/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:05:29.715310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:29.825811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:29.830015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15c1c-af8a-4c5a-bfaf-7f5980264f0e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:29.872786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:29.873472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:29.873758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:29.874037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:29.874772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:29.875086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:29.876499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:29.895129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 596/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:29.898844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:30.033294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af0db5-d7b1-400d-a367-63e18c0a7e35 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:30.034874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:30.075473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:30.076384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:30.076609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:30.076879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:30.086469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:30.088103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:30.106110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:30.107237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:30.108245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 616/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:30.116215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:30.238807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2e95b-2455-4cb0-9aff-0a62a300531d None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:30.278779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:30.279604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:30.279821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:30.280187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:30.281227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:30.281561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:30.283427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:30.304118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 597/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:30.311779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:30.425601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15fcf-c9f0-4f14-aad6-9c441c0a5ed7 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:30.465601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:30.466280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:30.466565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:30.466820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:30.499379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:30.542394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:30.544284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:30.657744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 617/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:05:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:05:30.663773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:30.690575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:30.790663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5f1d-9a01-44cb-bb5f-2c2e47faf808 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:30.831116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:30.834678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:30.834678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:30.834678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:30.844127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:30.845766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:30.867080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:30.868242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:30.869151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:30.869151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:30.869151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:30.869151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:30.869151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 598/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:05:30.874982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:30.946715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:30.996885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b07f3-6f7d-43b6-a373-afaa9ad7c0bb None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:31.040310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f219892-2da3-464e-97ee-89430434887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:31.041291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219892-2da3-464e-97ee-89430434887b None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:31.041685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219892-2da3-464e-97ee-89430434887b None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:31.042008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f219892-2da3-464e-97ee-89430434887b None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:31.042925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f219892-2da3-464e-97ee-89430434887b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:31.043279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f219892-2da3-464e-97ee-89430434887b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:31.045385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f219892-2da3-464e-97ee-89430434887b None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:31.059820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 618/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:31.066233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:31.156729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:31.202835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5ad58-19f2-4a38-ad70-a6c0e3c616e6 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:31.242311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:31.243148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:31.243436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:31.243711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:31.256137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:31.257629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:31.272787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:31.273785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:31.279520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 599/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:31.283255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f219892-2da3-464e-97ee-89430434887b None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:31.404786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f219892-2da3-464e-97ee-89430434887b None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:31.429287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:31.441569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:31.442284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:31.442568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:31.442890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:31.443741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:31.444056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:31.445492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caecfd1e-8851-49ad-842f-eae6e59d3153 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:31.463502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 619/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:31.465553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:31.596666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce7998-bc3e-4621-a8e1-987c36eaea46 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:31.630631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:31.638071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:31.638874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:31.639169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:31.639515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:31.640414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=fe214bb1d7bd4e6abd8256cab5ff579f) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:31.640782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:31.642366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a71ff-c1a6-4aec-9323-69f6c5053179 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:31.655286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 600/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:05:31 2022] GET /identity/v3/role_assignments?user.id=fe214bb1d7bd4e6abd8256cab5ff579f&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:33.788372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd49f45-97ea-4f83-a2d9-0cb2d9532bc6 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:33.788956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd49f45-97ea-4f83-a2d9-0cb2d9532bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:33.788956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd49f45-97ea-4f83-a2d9-0cb2d9532bc6 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:33.790401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 620/1220] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:33.796133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:33.796622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:33.797092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:33.819589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:33.820843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:33.896898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 601/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:33.920526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd49f45-97ea-4f83-a2d9-0cb2d9532bc6 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.042287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd49f45-97ea-4f83-a2d9-0cb2d9532bc6 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.079783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.080521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.080804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.081077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.091495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:34.099458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:34.100602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:34.101800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 621/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:05:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:34.106841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.229284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed2a3b-7f78-4afb-93e9-551b132fe35c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.254953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:34.267426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.268130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.268469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.268865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.270644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:34.270644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:34.271533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:34.281746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 602/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:34.288141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.340243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:34.403979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaf75f-019d-4de8-8881-f0c2ba24965c None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.439599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.440409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.440692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.440973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.452905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:34.454467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:34.474343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:34.475538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:34.476542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:34.476542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:34.476542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:34.476542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:34.476542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 622/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:34.482146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.592066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af449b-c6fc-4bea-acc1-5c48d16d60d0 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.626137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.626931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.627324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.627606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.628352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:34.628761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:34.630275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:34.643692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 603/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:34.650578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.748592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:34.756030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57039077-06af-45fa-93ec-e7942fc1c229 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.792526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.793415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.793772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.794089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.803810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:34.805479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:34.821366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:34.822660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:34.823910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 623/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:34.829386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:34.878717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:34.936664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8faffbc-64ba-4472-9ee4-c4743dcbf063 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:34.969204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:34.969875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:34.970155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:34.970423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:34.971293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:34.971598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:34.973401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:34.992034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 604/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:34.997932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:35.108138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5897421-cf10-4caa-910e-6b070bfcb12d None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:35.145873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:35.146720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:35.147058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:35.147412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:35.158805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:35.160678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:35.177842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:35.178979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:35.179936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:35.179936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:35.179936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:35.179936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:35.179936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 624/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:35.185733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:35.292347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e20225b-fcfc-41ea-a0b6-f188e05ab73e None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:35.295518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:35.333632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:35.334369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:35.334661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:35.334942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:35.335790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:35.336097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:35.337598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:35.355474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 605/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:35.356714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:35.366920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:35.464900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98777058-5db7-4c24-9cbd-e893c921826f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:35.499669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:35.500556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:35.500811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:35.501082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:35.507807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:35.509276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:35.524670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:35.525715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:35.526631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 625/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:35.531947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:35.647936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0040491-0ca7-43fb-b149-f5ea55d101d1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:35.687320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:35.688002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:35.688296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:35.688648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:35.689369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:35.689681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:35.691372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f491c-2070-41bf-a897-6bfc5d6bde32 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:35.704778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 606/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:35.710871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:35.778299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:35.819600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd9934-ea38-4114-955f-547271911eb5 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:35.860520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:35.861724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:35.862010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:35.862289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/fe214bb1d7bd4e6abd8256cab5ff579f/roles/cf5596c5cbaf4344bafe9f3957298776` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:35.879319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=fe214bb1d7bd4e6abd8256cab5ff579f, role_id=cf5596c5cbaf4344bafe9f3957298776)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:35.880802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448325c-e0b8-4ab4-8e2b-d112e8836a97 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:35.895408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 626/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:05:35 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/fe214bb1d7bd4e6abd8256cab5ff579f/roles/cf5596c5cbaf4344bafe9f3957298776 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:05:38.048925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7074b-1bdf-4e78-a185-2c605ac13d92 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.049527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7074b-1bdf-4e78-a185-2c605ac13d92 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.049527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e7074b-1bdf-4e78-a185-2c605ac13d92 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.054988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 607/1233] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:38.056632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.057038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.057194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.074312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:38.075561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:38.147891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 627/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:38.172350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7074b-1bdf-4e78-a185-2c605ac13d92 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:38.199181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:38.282421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e7074b-1bdf-4e78-a185-2c605ac13d92 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:38.326179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:38.327027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.327429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.327800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.341450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:38.350492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:38.351742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:38.352638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 608/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Thu Jun 30 04:05:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:38.358083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:38.406280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:38.464161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b8c1c-6b67-47fb-8d31-cbc46846eba5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:38.502790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:38.503577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.503959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.504243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.505002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:38.505311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:38.506715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:38.515522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 628/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Thu Jun 30 04:05:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:38.522123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:38.624310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:38.642871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec033f-1119-4045-8b51-7a5099a63908 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:38.684681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:38.685716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.686118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.686465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.699404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:38.700886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:38.719560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:38.720683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:38.721618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:38.721618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:38.721618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:38.721618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:38.721618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 609/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:38.727169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:38.841315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b22867-7f07-422e-aaab-515b83b22b65 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:38.850426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:38.876247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:38.876908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:38.877195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:38.877476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:38.878212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:38.878748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:38.880159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:38.892258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 629/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:38.898879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:39.012618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c9c66-f2cc-408e-abcf-12f01417bdb9 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:39.053387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:39.054238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:39.054539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:39.054839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:39.057769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:39.063544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:39.065633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:39.081038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:39.082354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:39.083574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 610/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:39.088615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:39.197097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e071802-5223-45c4-90d3-786c5dd7d6b0 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:39.236064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:39.237530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:39.237834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:39.238226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:39.239089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:39.239500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:39.241067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:39.259625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 630/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:39.266220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:39.388435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5b9a-6fe7-40e1-b094-893328ad6216 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:39.432402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:39.433474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:39.433951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:39.434427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:39.510205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:39.517962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:39.519738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:39.631422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 611/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:05:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:05:39.637335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:39.666673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:39.751884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2aa1ce-81a1-478c-8be6-afb9ed7b9679 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:39.786971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:39.787748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:39.788038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:39.788319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:39.802245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:39.804079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:39.824743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:39.825753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:39.826678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:39.826678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:39.826678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:39.826678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:39.826678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 631/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:05:39.833545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:39.953363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:39.960084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d1ed2-9ae2-408c-b973-31e7dbed56ae None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:39.998139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:39.998975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:39.999507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:39.999778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:40.000603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:40.000939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:40.002393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:40.015487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 612/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:40.022416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:40.109094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:40.133800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9c03-8bdc-435c-8671-99eff404314f None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:40.172360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:40.173192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:40.173517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:40.173870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:40.183033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:40.184548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:40.198370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:40.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:40.200525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 632/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:40.206200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:40.316059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220fcec-ed81-4fbb-b76d-6d79818d75cd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:40.352324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:40.353065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:40.353360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:40.353641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:40.354494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:40.354820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:40.356429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f3660-72c7-4d0d-b135-bec187a12b51 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:40.369012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 613/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:05:40.375555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:40.484248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20fc4e-a584-4f97-8598-bbfbe1218d0e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:40.522060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:40.522781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:40.523121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:40.523418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:40.524303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] RBAC: Adding query filter params (role.id=cf5596c5cbaf4344bafe9f3957298776, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=fe214bb1d7bd4e6abd8256cab5ff579f) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:40.524621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:40.526291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b018e8-fc97-4ef3-8493-b597fe49ed44 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:40.532948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 633/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:05:40 2022] GET /identity/v3/role_assignments?user.id=fe214bb1d7bd4e6abd8256cab5ff579f&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=cf5596c5cbaf4344bafe9f3957298776 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:05:42.836713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece07f8-6c2f-494c-bb45-5f64f39bd314 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:42.837275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece07f8-6c2f-494c-bb45-5f64f39bd314 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:42.837275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece07f8-6c2f-494c-bb45-5f64f39bd314 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:42.838891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 614/1247] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:42.844893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:42.845460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:42.845460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:42.851601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:42.866539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:42.867824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:42.944647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 634/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:42.969403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece07f8-6c2f-494c-bb45-5f64f39bd314 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:42.990478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:43.087261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece07f8-6c2f-494c-bb45-5f64f39bd314 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:43.130835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:43.131815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:43.132213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:43.132538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:43.143801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:43.151961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:43.153102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:43.154165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 615/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:05:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 04:05:43.161522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:43.265223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1766849-fb84-46f6-96e7-60d8f1982b48 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:43.298333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:43.305352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:43.306199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:43.306564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:43.306826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:43.307907 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:43.308260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:43.309946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:43.320762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 635/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:43.326717 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:43.416529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:43.441556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83eb3-4d56-4548-8efc-3788a342b318 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:43.476369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:43.477285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:43.477570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:43.477847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:43.489482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:43.490933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:43.508808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:43.509924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:43.511019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:43.511019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:43.511019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:43.511019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:43.511019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 616/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:43.516422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:43.623329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ba158-d1b3-4c4a-864f-a200ed6e9cc1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:43.662174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:43.662863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:43.664526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:43.664811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:43.665638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:43.666042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:43.667657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:43.682569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 636/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:43.688847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:43.805343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ec621-8ca0-4a14-8c47-8033afadab56 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:43.835495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:43.844407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:43.845745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:43.845745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:43.845745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:43.855520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:43.857115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:43.873709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:43.874926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:43.876062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:43.881012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 617/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:43.882189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:43.946978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:43.998018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaead80-15f5-4ad2-b08d-d04ee7f8d52e None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:44.036401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:44.037172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:44.037471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:44.037787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:44.038637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:44.038955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:44.040866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:44.062195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 637/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:05:44.068053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:44.173254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d69d0-bb3a-4d40-b510-979d496bce3f None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:44.209528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:44.210378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:44.210822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:44.211216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:44.278032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:44.294016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:44.295732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:44.427872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 618/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:05:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:05:44.429387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:44.538298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925864f-8a36-48e1-a7e6-f783bc9cfc4a None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:44.553953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:44.578255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:44.579114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:44.579404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:44.579677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:44.590910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:44.592843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:44.613650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:44.614797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:44.615924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:44.615924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:44.615924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:44.615924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:44.615924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 638/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:44.622320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:44.718435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:44.733345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939256c0-4696-4417-bad8-b952dfb24d00 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:44.776476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:44.777115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:44.777399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:44.777681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:44.778437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:44.778745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:44.780183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:44.792350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 619/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:44.798789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:44.905414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb565-c541-43cc-84d2-41a618b5e5dc None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:44.941710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:44.942533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:44.942816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:44.943122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:44.950901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:44.952767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:44.970594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:44.971753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:44.972989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 639/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:44.978319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:45.096334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59189a24-ce84-4584-8c37-d158a464b993 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:45.132156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:45.132826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:45.133107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:45.133382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:45.134157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:45.134465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:45.135931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b13c60-9eba-435e-85bf-759f056ad5ac None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:45.151133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 620/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:45.156054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:45.183598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:45.269997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fdbbe-8ef9-4c92-b76b-f4b92fd100d3 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:45.315419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:45.315419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:45.315866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:45.316085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:45.317104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=fe214bb1d7bd4e6abd8256cab5ff579f) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:45.317379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:45.319089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c61ed-fd8d-4441-b433-034653917ec2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:45.325433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 640/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:05:45 2022] GET /identity/v3/role_assignments?user.id=fe214bb1d7bd4e6abd8256cab5ff579f&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:47.356264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac820a3-25e3-435f-8d51-1739cd9ab4f0 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:47.356929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac820a3-25e3-435f-8d51-1739cd9ab4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:47.356929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac820a3-25e3-435f-8d51-1739cd9ab4f0 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:47.358199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 621/1261] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:47.363511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:47.363945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:47.364054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:47.382118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:47.383356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:47.454157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 641/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:47.477566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac820a3-25e3-435f-8d51-1739cd9ab4f0 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:47.493127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:47.589095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac820a3-25e3-435f-8d51-1739cd9ab4f0 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:47.625003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:47.625725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:47.626003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:47.626277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:47.637290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:47.645437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:47.646515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:47.647565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 622/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:05:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:47.655737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:47.682858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:47.770416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb88456-2857-4735-b94d-f7c16a5eabbd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:47.806225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:47.806897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:47.807209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:47.807488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:47.808240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:47.808550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:47.809954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:47.818300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 642/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:05:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:47.824377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:47.892267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:47.928144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7fd45-3260-440a-a6fc-62e09aa418eb None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:47.963436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:47.964217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:47.964497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:47.964773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:47.976836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:47.978287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:47.995627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:47.996713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:47.997777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:47.997777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:47.997777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:47.997777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:47.997777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 623/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:48.004908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.092598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:48.115318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591876cf-643d-4288-9f66-2d82d27eb451 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:48.150834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:48.151460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:48.151753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:48.152036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:48.152752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:48.153078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:48.154473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:48.167142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 643/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:48.173537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.283000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c5a12-6d70-4fe3-befa-0e0e6d961f36 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:48.311589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:48.336390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:48.337366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:48.337693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:48.338043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:48.348496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:48.350581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:48.369513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:48.371008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:48.372035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 624/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:48.376704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.480513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e02520-351b-4e1f-8801-8f699eff5c66 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:48.518688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:48.519412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:48.519863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:48.520366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:48.520983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:48.521289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:48.522730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:48.531566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:48.540234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 644/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:48.546207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.657006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6197fd-d03a-4349-85aa-0d555f8871f1 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:48.692445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:48.693244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:48.693522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:48.693803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:48.705666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:48.707269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:48.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:48.726847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:48.727827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:48.727827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:48.727827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:48.727827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:48.727827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 625/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:48.733301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.837345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076320-04d0-4195-b130-dae77d04f987 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:48.884790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:48.885451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:48.885731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:48.886029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:48.886749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:48.887050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:48.888450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:48.900743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 645/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:48.906725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:48.942396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:49.025569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8eed9-7df7-4e8d-9541-f68295bab725 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:49.060543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:49.061334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:49.061618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:49.061894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:49.068822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:49.070269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:49.084548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:49.085530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:49.086532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 626/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:49.093411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:49.154607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:49.205363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952c7a6-8856-4763-af33-3a443970eb82 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:49.241690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:49.242369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:49.242648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:49.242929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:49.243760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:49.244075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:49.245518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f932eca-fde1-448d-8e77-d9e3129f594f None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:49.257746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 646/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:49.264038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:49.363652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:49.372804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b0a2f-0710-4d46-85fb-af59ff86c159 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:49.410200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:49.411147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:49.411416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:49.411692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] PATH_INFO: `/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/fe214bb1d7bd4e6abd8256cab5ff579f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:49.413697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f78b240b6174aa8bdcbcdbdc26a36ce, user_id=fe214bb1d7bd4e6abd8256cab5ff579f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:49.415139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55683c-b8ec-4027-a628-7e1bb2f82c54 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:49.432116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 627/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Thu Jun 30 04:05:49 2022] PUT /identity/v3/projects/5f78b240b6174aa8bdcbcdbdc26a36ce/users/fe214bb1d7bd4e6abd8256cab5ff579f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:05:51.442261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50dcab-c571-4140-aba2-8be506ac702e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:51.443173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50dcab-c571-4140-aba2-8be506ac702e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:51.443173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50dcab-c571-4140-aba2-8be506ac702e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:51.445373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 647/1274] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:51.451638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:51.452364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:51.452513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:51.472927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:51.474184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:51.550574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 628/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:51.580718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50dcab-c571-4140-aba2-8be506ac702e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:51.687375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50dcab-c571-4140-aba2-8be506ac702e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:51.704780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:51.722802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:51.723617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:51.723898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:51.724176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:51.735421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 04:05:51.743040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:51.744141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:51.745610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 648/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Thu Jun 30 04:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:51.753940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:51.845029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:51.866044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70602c-36fc-4e4e-af14-637a311717c2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:51.902639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:51.903349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:51.903630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:51.903904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:51.904655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:51.904962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:51.906345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:51.915396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 629/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Thu Jun 30 04:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:51.921818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:52.031360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484bcb8-12e5-450c-b554-6050a75a7ba1 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:52.073216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:52.074373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:52.074750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:52.075150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:52.089040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:52.090521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:52.108159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:52.109190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:52.110440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:52.110440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:52.110440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:52.110440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:52.110440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 649/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:52.116343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:52.221890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9be6-0135-476e-95f8-76118ee18550 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:52.256386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:52.257041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:52.257329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:52.257603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:52.258337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:52.258605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:52.260002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:52.272774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 630/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:52.278782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:52.319832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:52.397727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1cf8a-130d-4c70-8813-224402158e62 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:52.434676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:52.435613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:52.435872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:52.436151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:52.444509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:52.445972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 04:05:52.460447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:52.461600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:52.462541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 650/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:52.467705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:52.471765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:52.587818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1fbe-2c87-4556-a6ff-45c808e5b40a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:52.624892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:52.625556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:52.625841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:52.626113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:52.626945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:52.627287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:52.628814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:52.646601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 631/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Thu Jun 30 04:05:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:52.652619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:52.749589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:52.762002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f00c-da2a-4a63-8272-25b9d2940be2 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:52.799699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:52.800349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:52.800672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:52.800947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:52.871035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:52.872431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:52.982238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:52.994058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 651/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Thu Jun 30 04:05:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 04:05:53.003504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:53.080487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:53.126754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79096aa8-4d34-4e97-b013-0f60ac70212c None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:53.161757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:53.162656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:53.163081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:53.163547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:53.177535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:53.179003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 04:05:53.202156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:53.203232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:53.208589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 04:05:53.208589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 04:05:53.208589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 04:05:53.208589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:53.208589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 632/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:53.210701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:53.320903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6fd87-bea6-4200-93a6-a29ad63ef1c4 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:53.360981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:53.361626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:53.361934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:53.362212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:53.362960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:53.363279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:53.364800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:53.377175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 652/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Thu Jun 30 04:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:53.383791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:53.500372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e85de7-999b-4bb3-a51b-692f51f635cd None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:53.515034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:53.540844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:53.541654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:53.541938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:53.542216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:53.550526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:53.552295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 04:05:53.567481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:53.568430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: ERROR keystone.server.flask.application Jun 30 04:05:53.569852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 633/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Thu Jun 30 04:05:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:53.574611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:53.628284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:53.678844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55ad5ce-e7fd-40b7-886b-922f60c8ccb8 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:53.715295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:53.715983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:53.716267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:53.716545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:53.717345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:53.717660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:53.719119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e499aa-568e-4024-977e-9f3e6c544cdb None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:53.731312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 653/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Thu Jun 30 04:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:53.737424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:53.854184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee50e4-9b27-4a7a-bf2f-b6981b193d1a None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:53.889400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:53.889986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:53.890273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:53.890559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:53.891423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] RBAC: Adding query filter params (role.id=7757cc59ef6443ada6dd11c520b3688b, scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce, user.id=fe214bb1d7bd4e6abd8256cab5ff579f) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:53.891747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:53.893195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfde0e2d-cf5d-49fb-a639-9b2fa94aef24 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:53.899220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 634/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Thu Jun 30 04:05:53 2022] GET /identity/v3/role_assignments?user.id=fe214bb1d7bd4e6abd8256cab5ff579f&scope.project.id=5f78b240b6174aa8bdcbcdbdc26a36ce&role.id=7757cc59ef6443ada6dd11c520b3688b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:55.976845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc450-abbc-41f0-8e80-d2f2eddb6cbe None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:55.977451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc450-abbc-41f0-8e80-d2f2eddb6cbe None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:55.977451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc450-abbc-41f0-8e80-d2f2eddb6cbe None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:55.978768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 654/1288] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:55.985081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:55.985542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:55.985645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:56.005917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:56.007145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:56.041468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:56.082290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 635/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:56.111909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bc450-abbc-41f0-8e80-d2f2eddb6cbe None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:56.225692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bc450-abbc-41f0-8e80-d2f2eddb6cbe None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:56.236503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:56.264908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:56.265707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:56.265989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:56.266260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:56.266916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:56.268336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:05:56.280936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:05:56.282436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:05:56.284575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 655/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:05:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:05:56.292666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:56.408489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17aa9c-0365-45f6-aa57-5e703c619360 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:56.445042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:56.445709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:56.445995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:56.446279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:56.446991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:56.447336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:56.448719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99772b56-c6ec-41eb-baa5-ca2fa401dfb9 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:56.453711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 636/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 30 04:05:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:56.460140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:56.566627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba5db3-12e6-4b20-bbfc-b4bdec497660 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:56.602061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:56.602733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:56.603017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:56.603333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] PATH_INFO: `/v3/services` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:56.604046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] RBAC: Adding query filter params (type=placement) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:05:56.604357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:56.605758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319af75c-7d9b-4e5e-883a-92667dac29ed None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:56.610419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 656/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 30 04:05:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:05:58.631956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774a8c-6fd0-420a-8918-d68e105ea49c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:58.632503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774a8c-6fd0-420a-8918-d68e105ea49c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:58.632503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc774a8c-6fd0-420a-8918-d68e105ea49c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:58.633909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 637/1293] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:05:58.640241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34582d-36eb-4e23-9ff1-9a04f5e6cfd3 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:58.640955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34582d-36eb-4e23-9ff1-9a04f5e6cfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:58.641155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34582d-36eb-4e23-9ff1-9a04f5e6cfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:58.660304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8e34582d-36eb-4e23-9ff1-9a04f5e6cfd3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:05:58.661505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8e34582d-36eb-4e23-9ff1-9a04f5e6cfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:05:58.692444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:58.736793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 657/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:05:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:05:58.773024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774a8c-6fd0-420a-8918-d68e105ea49c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:05:58.818947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:05:58.892727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc774a8c-6fd0-420a-8918-d68e105ea49c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:05:58.928533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:05:58.929198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:05:58.929483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:05:58.929765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:05:58.930485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:05:58.931901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb31f73-38fe-47d4-a930-f28f935928ed None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:05:58.950359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 638/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Thu Jun 30 04:05:58 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:06:01.012341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bd8f9-3540-4342-8eeb-40d374aa7045 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.013237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bd8f9-3540-4342-8eeb-40d374aa7045 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.013237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bd8f9-3540-4342-8eeb-40d374aa7045 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.015494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 658/1296] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:06:01.022036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.022484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.022591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.051108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:06:01.052313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:06:01.132980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 639/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:06:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 04:06:01.156142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bd8f9-3540-4342-8eeb-40d374aa7045 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:01.267639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bd8f9-3540-4342-8eeb-40d374aa7045 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:01.304822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:01.305983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.306421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.306830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.307826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:01.309717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:06:01.324904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:06:01.326022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:01.326957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 659/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:06:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:06:01.332683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:01.356097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:01.443017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cde797-1474-44e2-abca-8e5a58de16ba None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:01.477699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:01.478363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.478643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.478918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.479658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:06:01.479971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:01.481372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:01.486836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 640/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 30 04:06:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:06:01.493271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:01.527451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:01.602577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b51e0-0f7b-48bb-9781-4d363bbb4b19 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:01.637909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:01.638562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.638846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.639149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.639945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] RBAC: Adding query filter params (interface=public, service_id=47a85de63aae418c89ea066011db3e53, region_id=RegionOne) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:06:01.640261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:01.641642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dd3de6-d6be-4121-a043-5471a902c39c None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:01.646893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 660/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Thu Jun 30 04:06:01 2022] GET /identity/v3/endpoints?service_id=47a85de63aae418c89ea066011db3e53&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:06:01.652741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:01.749788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:01.769705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc84ca-2e77-46a5-aebc-550610c4aef3 None admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:01.805394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:01.805990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:01.806282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:01.806555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:01.807273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:01.808661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723353c5-79b8-45c2-bf50-d3ba2c0dd7e2 None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:01.816079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 641/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Thu Jun 30 04:06:01 2022] GET /identity/v3/services => generated 2191 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:06:03.828854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064aff4f-5895-4008-bcaf-3427bebb89ad None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:03.829732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064aff4f-5895-4008-bcaf-3427bebb89ad None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:03.829732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064aff4f-5895-4008-bcaf-3427bebb89ad None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:03.831933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 661/1302] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:06:03.840063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:03.840516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:03.840614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:03.858520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:06:03.859725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:06:03.930278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 642/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:06:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 04:06:03.953773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064aff4f-5895-4008-bcaf-3427bebb89ad None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:04.057943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:04.069030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064aff4f-5895-4008-bcaf-3427bebb89ad None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:04.109506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:04.110253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:04.110541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:04.110819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:04.111509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:04.112900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 04:06:04.125347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 04:06:04.126549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application with Lock( Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self._enter() Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application return self.creator() Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application created_value = creator( Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: ERROR keystone.server.flask.application Jun 30 04:06:04.127581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 662/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Thu Jun 30 04:06:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 04:06:04.133513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:04.216829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:04.241313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b26e7-31d2-4d92-9106-d049858846d9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:04.276563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:04.277225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:04.277507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:04.277778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] PATH_INFO: `/v3/services` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:04.278487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:06:04.278793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:04.280203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e5e9d-b623-4bd3-a34f-549b61a5075a None admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:04.285542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 643/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Thu Jun 30 04:06:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:06:04.291483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:06:04.408877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0eb05-8204-4f58-a7cd-aad3ddd7f775 None admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:06:04.447911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:06:04.448592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:06:04.448871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:06:04.449142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:06:04.449880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:06:04.451282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d7e742-0798-4aa3-a201-f52b753f4ee3 None admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:06:04.471865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 663/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Thu Jun 30 04:06:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:06:14.282275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=30 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:06:14.467290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:08.184506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4563a-5535-4e56-ae35-32c8b45fa6f7 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:08.184948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4563a-5535-4e56-ae35-32c8b45fa6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:08.185190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4563a-5535-4e56-ae35-32c8b45fa6f7 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:08.190589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 644/1307] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:08.192020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1722a-278c-4ea7-8275-8c39b532235b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:08.192435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1722a-278c-4ea7-8275-8c39b532235b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:08.192640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1722a-278c-4ea7-8275-8c39b532235b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:08.355212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d1a1722a-278c-4ea7-8275-8c39b532235b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:08.356432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d1a1722a-278c-4ea7-8275-8c39b532235b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:08.508383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 664/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:08.560153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:08.560538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:08.560922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:08.562510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 645/1309] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jun 30 04:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:08.572769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804bef-c8b5-4231-a10b-65418c3d8007 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:08.573058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804bef-c8b5-4231-a10b-65418c3d8007 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:08.573337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e804bef-c8b5-4231-a10b-65418c3d8007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:08.665120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8e804bef-c8b5-4231-a10b-65418c3d8007 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:08.667367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8e804bef-c8b5-4231-a10b-65418c3d8007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:08.696696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:08.779434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 665/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jun 30 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:08.791835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:08.794270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:08.934729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39512b-575d-483a-96f1-dfc5a386a438 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:09.022816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:09.023517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:09.023803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:09.024066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:09.025735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:09.083341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:09.143002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:09.144489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f6b15-6f99-467b-8838-c09b92c0981f service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:09.164852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 646/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:11.577957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:11.578553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:11.578553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:11.584190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 666/1312] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:11.586331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c510c6-5442-4678-889a-fad3a1ebcefb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:11.586811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c510c6-5442-4678-889a-fad3a1ebcefb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:11.586905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c510c6-5442-4678-889a-fad3a1ebcefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:11.665648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b3c510c6-5442-4678-889a-fad3a1ebcefb None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:11.666903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b3c510c6-5442-4678-889a-fad3a1ebcefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:11.782418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 647/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:11.822302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:11.823902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:11.935534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eae81-c857-4dcc-a3ce-58bc49cdec1b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:12.010167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:12.012757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:12.013516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:12.013856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:12.014111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:12.016953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:12.134370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:12.135676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ceb7f-6986-4837-b8a6-4832323b9306 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:12.154513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 667/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:14.499874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:14.500467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:14.500467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:14.502033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 648/1315] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:14.508394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7fe6c-38b5-4613-a069-c5f11f159c0c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:14.509174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7fe6c-38b5-4613-a069-c5f11f159c0c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:14.509326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7fe6c-38b5-4613-a069-c5f11f159c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:14.610120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-07c7fe6c-38b5-4613-a069-c5f11f159c0c None None] MFA Rules not processed for user `c461c48ade384f048bfd341c19ff0197`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:14.611462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-07c7fe6c-38b5-4613-a069-c5f11f159c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:14.667058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:14.714270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 668/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:14.766485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:14.768073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:14.779417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:14.877830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43562172-da17-4dce-b1cb-82f731636a74 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:14.975018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:14.976038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:14.976470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:14.976891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:14.979474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:15.052823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:15.101095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:15.102629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca833483-40c1-4f19-ba43-c43020e71d4e service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:15.121319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 649/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:17.734572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:17.735199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:17.735199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:17.736542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 669/1318] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:17.743124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51f5ff-f258-4b29-9ae1-1a02fe611f0a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:17.743566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51f5ff-f258-4b29-9ae1-1a02fe611f0a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:17.743784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51f5ff-f258-4b29-9ae1-1a02fe611f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:17.824977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-fb51f5ff-f258-4b29-9ae1-1a02fe611f0a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:17.826123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fb51f5ff-f258-4b29-9ae1-1a02fe611f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:17.934334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 650/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:17.983851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:17.985418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:18.100625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ece6-8686-47a4-b5db-a7093839dc88 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:18.178136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:18.178795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:18.179083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:18.179393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:18.181036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:18.188571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:18.305203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:18.307155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205fb13a-acfe-4725-83f7-c5adbd7c9efb service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:18.330035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 670/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:07:21.056516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aed031-8ab7-44ab-bcda-12f6f9dd3d9a None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:21.057084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aed031-8ab7-44ab-bcda-12f6f9dd3d9a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:21.057084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aed031-8ab7-44ab-bcda-12f6f9dd3d9a None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:21.058487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 651/1321] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:21.063916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aae17-c265-4eba-9c24-52035647e06c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:21.064353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aae17-c265-4eba-9c24-52035647e06c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:21.064472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841aae17-c265-4eba-9c24-52035647e06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:21.132996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-841aae17-c265-4eba-9c24-52035647e06c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:21.134223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-841aae17-c265-4eba-9c24-52035647e06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:21.195957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:21.240600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 671/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:21.293300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:21.293748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:21.294107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:21.296435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 652/1323] 10.222.0.13 () {54 vars in 964 bytes} [Thu Jun 30 04:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:21.303243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9fb0-a07d-4bde-8bc6-7c92f06186d4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:21.303559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9fb0-a07d-4bde-8bc6-7c92f06186d4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:21.304042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba9fb0-a07d-4bde-8bc6-7c92f06186d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:21.391958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-95ba9fb0-a07d-4bde-8bc6-7c92f06186d4 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:21.393251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-95ba9fb0-a07d-4bde-8bc6-7c92f06186d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:21.472102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 672/1324] 10.222.0.13 () {58 vars in 1061 bytes} [Thu Jun 30 04:07:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:21.481390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:21.482987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:21.503030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:21.594682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ea057-9064-4e60-8c2c-d83350bbb832 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:21.669709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:21.670312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:21.670597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:21.670882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:21.672536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:21.749181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:21.788455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:21.789898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32dde7-4cde-485a-9415-ae8a63e449b3 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:21.808991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 653/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:24.120406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:24.120984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:24.120984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:24.122567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 673/1326] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:24.128489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31ea33-1aec-4080-9548-895a7b3764b9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:24.128994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31ea33-1aec-4080-9548-895a7b3764b9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:24.129086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31ea33-1aec-4080-9548-895a7b3764b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:24.199124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6a31ea33-1aec-4080-9548-895a7b3764b9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:24.200392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6a31ea33-1aec-4080-9548-895a7b3764b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:24.310634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 654/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:24.346674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:24.348455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:24.414861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:24.460338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60c90b-bb94-42de-bf4b-bcc593cdb987 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:24.538155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:24.538817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:24.539220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:24.539508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:24.541155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:24.655920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:24.658821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:24.660355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bccd87-8679-4e9a-8e49-da87d861eb3e service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:24.682544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 674/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:26.802684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:26.803642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:26.804324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:26.805756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 655/1329] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:26.811688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18990b-4a3b-477f-98c4-a23bf61f2d86 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:26.812189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18990b-4a3b-477f-98c4-a23bf61f2d86 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:26.812288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18990b-4a3b-477f-98c4-a23bf61f2d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:26.885138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ed18990b-4a3b-477f-98c4-a23bf61f2d86 None None] MFA Rules not processed for user `c461c48ade384f048bfd341c19ff0197`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:26.886653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ed18990b-4a3b-477f-98c4-a23bf61f2d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:27.008351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 675/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:27.046925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:27.048507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:27.082787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:27.169741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e7132-da20-4840-b5dc-93fd0162fa1f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:27.264778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:27.265355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:27.265643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:27.265922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:27.267595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:27.350040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:27.383825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:27.385290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ba26b-4942-4c7b-b9e8-c0a3af641728 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:27.406286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 656/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:29.860210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:29.861033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:29.861033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:29.863186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 676/1332] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:29.872201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dde52b-65a1-43e6-bd6f-1d419aceaf16 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:29.872646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dde52b-65a1-43e6-bd6f-1d419aceaf16 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:29.872750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dde52b-65a1-43e6-bd6f-1d419aceaf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:29.944611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-91dde52b-65a1-43e6-bd6f-1d419aceaf16 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:29.945828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-91dde52b-65a1-43e6-bd6f-1d419aceaf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:30.050762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 657/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:30.092983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:30.094651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:30.169112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:30.204449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700fe19-34a3-49c1-90de-c35881c5b727 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:30.281479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:30.282142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:30.282423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:30.282695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:30.284364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:30.401364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:30.402710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4155fa1-4bf4-4a9b-8cc9-835ada9937af service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:30.413441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:30.424179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 677/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:32.463170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:32.464014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:32.464182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:32.466619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 658/1335] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:32.474966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad22359-f750-40d7-baa9-d124eab43b0d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:32.475482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad22359-f750-40d7-baa9-d124eab43b0d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:32.475581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad22359-f750-40d7-baa9-d124eab43b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:32.555140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9ad22359-f750-40d7-baa9-d124eab43b0d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:32.556401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9ad22359-f750-40d7-baa9-d124eab43b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:32.663565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 678/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:32.719068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:32.721484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:32.766246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:32.837236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cc866-4d79-4f8f-9bf8-efab262ad9f9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:32.912002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:32.912658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:32.912977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:32.913255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:32.914897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:33.007862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:33.037644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:33.039601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055d148-a7e4-407f-817e-d4209e5842e3 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:33.058168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 659/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:35.560306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:35.560873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:35.560873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:35.562248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 679/1338] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:35.567777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971d5e4-e231-43f1-8e20-4c1ffadbe2be None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:35.568210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971d5e4-e231-43f1-8e20-4c1ffadbe2be None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:35.568315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971d5e4-e231-43f1-8e20-4c1ffadbe2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:35.644530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7971d5e4-e231-43f1-8e20-4c1ffadbe2be None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:35.645757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7971d5e4-e231-43f1-8e20-4c1ffadbe2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:35.762234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 660/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:35.801653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:35.803250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:35.884182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:35.913446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ac895-133c-4cc8-85f4-f9e972ccc575 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:35.991952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:35.992643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:35.992928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:35.993201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:35.994855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:36.120360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:36.121809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dddbb-a2b3-4e14-b902-fedff5625e3d service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:36.140282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 680/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:07:40.235212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:40.235769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:40.235769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:40.237335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 661/1341] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:40.242741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac324f4-b9dd-474d-9b3c-6a12d2b0aac1 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:40.243244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac324f4-b9dd-474d-9b3c-6a12d2b0aac1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:40.243414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac324f4-b9dd-474d-9b3c-6a12d2b0aac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:40.250566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:40.317379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9ac324f4-b9dd-474d-9b3c-6a12d2b0aac1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:40.318589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9ac324f4-b9dd-474d-9b3c-6a12d2b0aac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:40.441001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 681/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:40.480883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:40.483358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:40.535847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:40.590165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d97c0-4781-43c7-ba32-d06f105f0a04 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:40.594036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:40.594699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:40.594984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:40.595291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:40.604832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:40.606278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9c1e4-38e3-43db-81fd-f3358e280a47 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:40.615643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 662/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:43.288373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:43.288946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:43.289056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:43.290469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 682/1344] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:43.296641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b4ce-4d30-4de6-b3dd-b9e1c253e291 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:43.297112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b4ce-4d30-4de6-b3dd-b9e1c253e291 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:43.297309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784b4ce-4d30-4de6-b3dd-b9e1c253e291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:43.328492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d784b4ce-4d30-4de6-b3dd-b9e1c253e291 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:43.329686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d784b4ce-4d30-4de6-b3dd-b9e1c253e291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:43.407356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 663/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:43.445612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:43.447787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:43.463031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d037aa-59c3-4f9f-80cd-a65a01d93923 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:43.466843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:43.467587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:43.467874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:43.468155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:43.478757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:43.480274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905aaf3-df1e-4824-bc99-ac9a5416cabf service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:43.491004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 683/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:47.503935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372eac6-a3c8-4c27-90fb-49843c7ff914 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:47.504493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372eac6-a3c8-4c27-90fb-49843c7ff914 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:47.504493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372eac6-a3c8-4c27-90fb-49843c7ff914 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:47.505913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 664/1347] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:47.511414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844465-73aa-484b-b662-6bdb3b626203 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:47.511919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844465-73aa-484b-b662-6bdb3b626203 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:47.511919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02844465-73aa-484b-b662-6bdb3b626203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:47.530502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-02844465-73aa-484b-b662-6bdb3b626203 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:47.531866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-02844465-73aa-484b-b662-6bdb3b626203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:47.540551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 684/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:07:47.579137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372eac6-a3c8-4c27-90fb-49843c7ff914 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:07:47.593784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372eac6-a3c8-4c27-90fb-49843c7ff914 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:07:47.597548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:07:47.598206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:47.598489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:47.598771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:47.608757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:07:47.610670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690be5d-dee5-4d83-bb5e-dd3cccd202df service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:07:47.622277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 665/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:07:52.630296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404e48f-a18d-47c9-bac4-22f555ff0b71 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:52.631220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404e48f-a18d-47c9-bac4-22f555ff0b71 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:52.631781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404e48f-a18d-47c9-bac4-22f555ff0b71 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:52.636976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 685/1350] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:07:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:52.649039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0c39b-c120-49ce-bbee-c2da7edc8c6d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:52.649500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0c39b-c120-49ce-bbee-c2da7edc8c6d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:52.649600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb0c39b-c120-49ce-bbee-c2da7edc8c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:52.766176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8eb0c39b-c120-49ce-bbee-c2da7edc8c6d None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:52.767585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8eb0c39b-c120-49ce-bbee-c2da7edc8c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:52.797991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:52.866224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 666/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:07:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:57.231710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcd3d-400d-474f-9deb-fcac843d0e82 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:57.232363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcd3d-400d-474f-9deb-fcac843d0e82 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:57.232363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cbcd3d-400d-474f-9deb-fcac843d0e82 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:57.233600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 686/1352] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:07:57.238842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eef5c-9ef0-4d9b-82fa-4da2dafe99dd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:57.239327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eef5c-9ef0-4d9b-82fa-4da2dafe99dd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:57.239416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183eef5c-9ef0-4d9b-82fa-4da2dafe99dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:57.258123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-183eef5c-9ef0-4d9b-82fa-4da2dafe99dd None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:57.259443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-183eef5c-9ef0-4d9b-82fa-4da2dafe99dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:57.268198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 667/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:07:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:07:57.525224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=72 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:07:58.747228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f5a1-3f21-402d-a032-aec74457ff36 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:58.747884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f5a1-3f21-402d-a032-aec74457ff36 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:58.747884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f5a1-3f21-402d-a032-aec74457ff36 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:58.749146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 687/1354] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:07:58.754312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4ff3-0bdb-4d1b-af2d-d9ba25436662 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:07:58.754724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4ff3-0bdb-4d1b-af2d-d9ba25436662 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:07:58.754867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa4ff3-0bdb-4d1b-af2d-d9ba25436662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:07:58.773772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-bffa4ff3-0bdb-4d1b-af2d-d9ba25436662 None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:07:58.774977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bffa4ff3-0bdb-4d1b-af2d-d9ba25436662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:07:58.783899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 668/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:07:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:02.270687 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da712-1907-49c4-8f81-fc156705fd58 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:02.279539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da712-1907-49c4-8f81-fc156705fd58 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:02.279539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da712-1907-49c4-8f81-fc156705fd58 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:02.279539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 688/1356] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:02.281251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b902-ce10-4f8e-8aed-1ff22c3574bd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:02.281697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b902-ce10-4f8e-8aed-1ff22c3574bd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:02.281822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b902-ce10-4f8e-8aed-1ff22c3574bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:02.304618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5678b902-ce10-4f8e-8aed-1ff22c3574bd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:02.306419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5678b902-ce10-4f8e-8aed-1ff22c3574bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:02.314970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 669/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:02.372360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ae21-ff33-43c0-9595-7ba3f0cbd29a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:02.373007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ae21-ff33-43c0-9595-7ba3f0cbd29a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:02.374233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676ae21-ff33-43c0-9595-7ba3f0cbd29a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:02.377438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 689/1358] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 30 04:08:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:02.386188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a048d86-a1a0-46ce-a08e-6b24dff2d31f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:02.386447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a048d86-a1a0-46ce-a08e-6b24dff2d31f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:02.390500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a048d86-a1a0-46ce-a08e-6b24dff2d31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:02.443489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6a048d86-a1a0-46ce-a08e-6b24dff2d31f None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:02.445290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6a048d86-a1a0-46ce-a08e-6b24dff2d31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:02.531398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 670/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 30 04:08:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:02.539186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ae21-ff33-43c0-9595-7ba3f0cbd29a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:02.555794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676ae21-ff33-43c0-9595-7ba3f0cbd29a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:02.561360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:02.562013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:02.562295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:02.562571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:02.578206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:02.580450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979ccef-c2f7-455a-8f05-21807cc0ca64 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:02.592413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 690/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:04.824744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe9e9-fca7-43f1-8b59-fe298dc871c8 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:04.825322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe9e9-fca7-43f1-8b59-fe298dc871c8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:04.825322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fe9e9-fca7-43f1-8b59-fe298dc871c8 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:04.826873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 671/1361] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:04.832097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77547-15ce-4d33-95e2-1a5a5634ca65 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:04.832546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77547-15ce-4d33-95e2-1a5a5634ca65 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:04.832640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d77547-15ce-4d33-95e2-1a5a5634ca65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:04.852924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c6d77547-15ce-4d33-95e2-1a5a5634ca65 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:04.853757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c6d77547-15ce-4d33-95e2-1a5a5634ca65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:04.861715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 691/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:04.907352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe9e9-fca7-43f1-8b59-fe298dc871c8 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:04.923427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fe9e9-fca7-43f1-8b59-fe298dc871c8 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:04.930234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:04.930882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:04.931189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:04.931469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:04.940868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:04.942470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb7eb0-b7a1-441a-acc1-6c4b438fa941 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:04.953625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 672/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:07.336736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e2e1-efdb-417a-b424-360a118ac644 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:07.337625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e2e1-efdb-417a-b424-360a118ac644 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:07.338056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217e2e1-efdb-417a-b424-360a118ac644 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:07.339941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 692/1364] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:07.345360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690869f-3a71-4c91-8c6f-b511062a5a55 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:07.345830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690869f-3a71-4c91-8c6f-b511062a5a55 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:07.346057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690869f-3a71-4c91-8c6f-b511062a5a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:07.367734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a690869f-3a71-4c91-8c6f-b511062a5a55 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:07.369029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a690869f-3a71-4c91-8c6f-b511062a5a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:07.376950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 673/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:07.423285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0c180-1f28-482b-a732-0c0c4b6ecfdc None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:07.423651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0c180-1f28-482b-a732-0c0c4b6ecfdc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:07.424026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0c180-1f28-482b-a732-0c0c4b6ecfdc None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:07.425428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 693/1366] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 30 04:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:07.431126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe722-1c4c-4eac-be32-1ca3098f122a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:07.431469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe722-1c4c-4eac-be32-1ca3098f122a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:07.431753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbe722-1c4c-4eac-be32-1ca3098f122a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:07.452869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9afbe722-1c4c-4eac-be32-1ca3098f122a None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:07.454068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9afbe722-1c4c-4eac-be32-1ca3098f122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:07.465722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 674/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 30 04:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:07.473607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0c180-1f28-482b-a732-0c0c4b6ecfdc None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:07.491553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0c180-1f28-482b-a732-0c0c4b6ecfdc None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:07.497199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:07.497857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:07.498144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:07.498420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:07.506980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:07.508702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fb8b7-24fa-4b69-bd64-18fdbef0fca4 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:07.518057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 694/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:09.614706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b038e1-e970-401d-90e8-c2e036aeb43f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:09.615368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b038e1-e970-401d-90e8-c2e036aeb43f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:09.615368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b038e1-e970-401d-90e8-c2e036aeb43f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:09.617184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 675/1369] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:09.623300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbac2-8a30-4976-a243-70f5c8fb55fc None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:09.623908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbac2-8a30-4976-a243-70f5c8fb55fc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:09.623908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefbac2-8a30-4976-a243-70f5c8fb55fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:09.643143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-faefbac2-8a30-4976-a243-70f5c8fb55fc None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:09.644393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-faefbac2-8a30-4976-a243-70f5c8fb55fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:09.653182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 695/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:09.697323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b038e1-e970-401d-90e8-c2e036aeb43f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:09.713512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b038e1-e970-401d-90e8-c2e036aeb43f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:09.719691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:09.721692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:09.722066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:09.722413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:09.731661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:09.733356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02091346-f01e-4fc3-bbc8-22b09ba9d633 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:09.743585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 676/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:10.324954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ac52-5607-48e3-89bf-296d63f51155 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:10.325244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ac52-5607-48e3-89bf-296d63f51155 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:10.325703 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7964ac52-5607-48e3-89bf-296d63f51155 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:10.327089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 696/1372] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:08:10.332413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167304-dbf8-4cfd-b6fb-3880e78da2b3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:10.332890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167304-dbf8-4cfd-b6fb-3880e78da2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:10.333322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0167304-dbf8-4cfd-b6fb-3880e78da2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:10.351968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e0167304-dbf8-4cfd-b6fb-3880e78da2b3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:10.353456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e0167304-dbf8-4cfd-b6fb-3880e78da2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:10.362200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 677/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:10.671048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf622e5-ffa7-413c-b2f6-93c6a654237b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:10.671530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf622e5-ffa7-413c-b2f6-93c6a654237b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:10.671612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf622e5-ffa7-413c-b2f6-93c6a654237b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:10.672978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 697/1374] 10.222.0.13 () {54 vars in 963 bytes} [Thu Jun 30 04:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:10.678083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d835fe-dbf5-46d6-89f9-2d138e355d35 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:10.678653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d835fe-dbf5-46d6-89f9-2d138e355d35 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:10.678653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d835fe-dbf5-46d6-89f9-2d138e355d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:10.685761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:08:10.731052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-38d835fe-dbf5-46d6-89f9-2d138e355d35 None None] MFA Rules not processed for user `d534d56b90834341afb915c39e70faff`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:10.732304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-38d835fe-dbf5-46d6-89f9-2d138e355d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:10.793618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 678/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Jun 30 04:08:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:10.801560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf622e5-ffa7-413c-b2f6-93c6a654237b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:10.813218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf622e5-ffa7-413c-b2f6-93c6a654237b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:10.817105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:10.817989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:10.818370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:10.818751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:10.828751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:10.830331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de6e-2170-4f1c-9ba3-43e716cd021d service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:10.841473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 698/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:11.878560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f040-3c18-435e-bd1e-4048063c1d2a None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:11.879146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f040-3c18-435e-bd1e-4048063c1d2a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:11.879146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17f040-3c18-435e-bd1e-4048063c1d2a None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:11.880625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 679/1377] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:11.886090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5e8f-fdba-42e9-9778-b79328614221 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:11.886538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5e8f-fdba-42e9-9778-b79328614221 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:11.886654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b5e8f-fdba-42e9-9778-b79328614221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:11.905237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c67b5e8f-fdba-42e9-9778-b79328614221 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:11.906437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c67b5e8f-fdba-42e9-9778-b79328614221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:11.915569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 699/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:11.964155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17f040-3c18-435e-bd1e-4048063c1d2a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:11.979961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17f040-3c18-435e-bd1e-4048063c1d2a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:11.986170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:11.986833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:11.987141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:11.987427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:11.996774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:11.998394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5142b3-1a3e-4389-b7b7-bd26d9dc753e service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:12.008791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 680/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:14.152479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6882-b701-4d6d-96c4-046d9815f919 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:14.153041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6882-b701-4d6d-96c4-046d9815f919 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:14.153041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff6882-b701-4d6d-96c4-046d9815f919 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:14.154466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 700/1380] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:14.160354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ebc1-d96f-4fdd-a1ca-ff5da66a3d02 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:14.160841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ebc1-d96f-4fdd-a1ca-ff5da66a3d02 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:14.160942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309ebc1-d96f-4fdd-a1ca-ff5da66a3d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:14.182328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b309ebc1-d96f-4fdd-a1ca-ff5da66a3d02 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:14.183639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b309ebc1-d96f-4fdd-a1ca-ff5da66a3d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:14.193231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 681/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:14.238376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff6882-b701-4d6d-96c4-046d9815f919 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:14.254349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff6882-b701-4d6d-96c4-046d9815f919 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:14.260685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:14.261375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:14.261691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:14.262034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:14.270991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:14.272720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99a8c9-498d-4fa2-aa50-7c1f8d93ea23 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:14.282324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 701/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:16.372790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef454f-7f98-45ef-8f81-e61720cc95a5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:16.373327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef454f-7f98-45ef-8f81-e61720cc95a5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:16.373327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef454f-7f98-45ef-8f81-e61720cc95a5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:16.374626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 682/1383] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:08:16.379982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770af1-ccbb-474f-97d7-6ea95855ea6d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:16.380452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770af1-ccbb-474f-97d7-6ea95855ea6d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:16.380546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770af1-ccbb-474f-97d7-6ea95855ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:16.399086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-05770af1-ccbb-474f-97d7-6ea95855ea6d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:16.400320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-05770af1-ccbb-474f-97d7-6ea95855ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:16.408635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 702/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:16.454181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef454f-7f98-45ef-8f81-e61720cc95a5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:16.471407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef454f-7f98-45ef-8f81-e61720cc95a5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:16.477621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:16.478527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:16.478925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:16.479352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:16.489929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:16.491981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710ba2c-ffbd-4fda-b310-03a8d5b51033 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:16.514520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 683/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:18.556969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6a04-a71c-479c-8695-6ba4ac68213e None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:18.557664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6a04-a71c-479c-8695-6ba4ac68213e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:18.557664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c6a04-a71c-479c-8695-6ba4ac68213e None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:18.558896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 703/1386] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:18.564456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510564c-b075-4ae7-8cdd-5e365f2f81d0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:18.564948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510564c-b075-4ae7-8cdd-5e365f2f81d0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:18.565028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510564c-b075-4ae7-8cdd-5e365f2f81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:18.586193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9510564c-b075-4ae7-8cdd-5e365f2f81d0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:18.587506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9510564c-b075-4ae7-8cdd-5e365f2f81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:18.597280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 684/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:08:18.645827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6a04-a71c-479c-8695-6ba4ac68213e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:18.663334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c6a04-a71c-479c-8695-6ba4ac68213e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:18.671524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:18.672479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:18.672865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:18.673233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:18.685417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:18.687628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085383-e3a7-40a5-9848-c880ed5a2176 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:18.701519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 704/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:20.874132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9357-a3ff-4ebe-925b-10fd26de76c7 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:20.874734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9357-a3ff-4ebe-925b-10fd26de76c7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:20.874887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9357-a3ff-4ebe-925b-10fd26de76c7 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:20.877013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 685/1389] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:20.882504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fc6d-f03f-4f84-b43f-1b17dfcf0cfd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:20.883121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fc6d-f03f-4f84-b43f-1b17dfcf0cfd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:20.883121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623fc6d-f03f-4f84-b43f-1b17dfcf0cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:20.902704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0623fc6d-f03f-4f84-b43f-1b17dfcf0cfd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:20.904107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0623fc6d-f03f-4f84-b43f-1b17dfcf0cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:20.912725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 705/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:20.972005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c9357-a3ff-4ebe-925b-10fd26de76c7 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:20.988586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c9357-a3ff-4ebe-925b-10fd26de76c7 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:20.994814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:20.995536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:20.995841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:20.996134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:21.005223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:21.007003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664767-b7b4-42d1-8bc4-4e4faf50fd3c service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:21.017817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 686/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:23.167052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e857b-b176-4fe1-a13c-39287c4ba96f None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:23.167679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e857b-b176-4fe1-a13c-39287c4ba96f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:23.167679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e857b-b176-4fe1-a13c-39287c4ba96f None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:23.169880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 706/1392] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:23.175581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ea4d-0b53-4ccc-a264-9043a7687495 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:23.176181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ea4d-0b53-4ccc-a264-9043a7687495 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:23.176181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087ea4d-0b53-4ccc-a264-9043a7687495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:23.197674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a087ea4d-0b53-4ccc-a264-9043a7687495 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:23.199431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a087ea4d-0b53-4ccc-a264-9043a7687495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:23.211773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 687/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:23.269584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e857b-b176-4fe1-a13c-39287c4ba96f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:23.286887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e857b-b176-4fe1-a13c-39287c4ba96f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:23.292849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:23.293578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:23.294213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:23.294505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:23.304980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:23.306591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c44810-16fd-410b-98cf-bf60941d8c25 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:23.317138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 707/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:25.417441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec5638-d801-45c4-94da-426fe73fea3b None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:25.418014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec5638-d801-45c4-94da-426fe73fea3b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:25.418014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec5638-d801-45c4-94da-426fe73fea3b None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:25.419784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 688/1395] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:25.425073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2530f-5f33-4436-a3a1-d785b9d19912 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:25.425496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2530f-5f33-4436-a3a1-d785b9d19912 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:25.425605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2530f-5f33-4436-a3a1-d785b9d19912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:25.443860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cfa2530f-5f33-4436-a3a1-d785b9d19912 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:25.445179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cfa2530f-5f33-4436-a3a1-d785b9d19912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:25.453323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 708/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:25.497563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec5638-d801-45c4-94da-426fe73fea3b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:25.512667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec5638-d801-45c4-94da-426fe73fea3b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:25.518252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:25.518923 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:25.519237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:25.519514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:25.528142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:25.529735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d9b191-5fb6-4bcc-b29d-e85f172896b7 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:25.539614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 689/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:26.057200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b59fa-2b1d-4a9c-9fbe-81fff4f9e023 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:26.057447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b59fa-2b1d-4a9c-9fbe-81fff4f9e023 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:26.057746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b59fa-2b1d-4a9c-9fbe-81fff4f9e023 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:26.059120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 709/1398] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:26.064389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf60b-eaa8-4deb-a07a-68384c8aaa3e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:26.064685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf60b-eaa8-4deb-a07a-68384c8aaa3e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:26.064960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cdf60b-eaa8-4deb-a07a-68384c8aaa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:26.083130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-45cdf60b-eaa8-4deb-a07a-68384c8aaa3e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:26.084337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-45cdf60b-eaa8-4deb-a07a-68384c8aaa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:26.093283 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 690/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:26.150419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce09a9-9e9d-458f-a1bc-0cb2ed68d634 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:26.150713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce09a9-9e9d-458f-a1bc-0cb2ed68d634 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:26.150992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fce09a9-9e9d-458f-a1bc-0cb2ed68d634 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:26.152384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 710/1400] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jun 30 04:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:08:26.157472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c104-4b36-4df9-bf1f-6ce297883200 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:26.157865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c104-4b36-4df9-bf1f-6ce297883200 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:26.158140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57c104-4b36-4df9-bf1f-6ce297883200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:26.209795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2f57c104-4b36-4df9-bf1f-6ce297883200 None None] MFA Rules not processed for user `aecdfa0fc8514b09b04bd7ee759d21ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:26.211402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2f57c104-4b36-4df9-bf1f-6ce297883200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:26.246009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:08:26.275112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 691/1401] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jun 30 04:08:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:26.282833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce09a9-9e9d-458f-a1bc-0cb2ed68d634 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:26.294154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fce09a9-9e9d-458f-a1bc-0cb2ed68d634 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:26.297854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:26.298834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:26.299220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:26.299593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:26.310957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:26.312555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de2699-d214-4449-840f-6e14902ac260 service glance] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:26.322575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 711/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:27.047518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072cdb3-838c-4550-9074-fa261681c678 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.047991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072cdb3-838c-4550-9074-fa261681c678 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.048095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072cdb3-838c-4550-9074-fa261681c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.098019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0072cdb3-838c-4550-9074-fa261681c678 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:27.099248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0072cdb3-838c-4550-9074-fa261681c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:27.164187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 692/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:27.211802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7919-245f-4b05-ba02-c14950d567ac None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.212298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7919-245f-4b05-ba02-c14950d567ac None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.212410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b7919-245f-4b05-ba02-c14950d567ac None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.213776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 712/1404] 10.222.0.13 () {54 vars in 942 bytes} [Thu Jun 30 04:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:27.220498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca913f-f944-43a1-96cc-e5b848dc04c9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.220798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca913f-f944-43a1-96cc-e5b848dc04c9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.221071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca913f-f944-43a1-96cc-e5b848dc04c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.270197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-47ca913f-f944-43a1-96cc-e5b848dc04c9 None None] MFA Rules not processed for user `0857fc0c4021456b88288ba10921ca6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:27.271358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-47ca913f-f944-43a1-96cc-e5b848dc04c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:27.347973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 693/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Thu Jun 30 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:27.358080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b7919-245f-4b05-ba02-c14950d567ac None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:27.372420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b7919-245f-4b05-ba02-c14950d567ac None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:27.377690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:27.378328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.378612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.378902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.387904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:27.389486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe36cb6-796e-42be-a863-739edff6ecaa service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:27.397560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 713/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:08:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:08:27.625721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691e6b6-397b-4402-9fdc-580edfd4e865 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.626011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691e6b6-397b-4402-9fdc-580edfd4e865 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.626297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8691e6b6-397b-4402-9fdc-580edfd4e865 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.628150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 694/1407] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:27.633978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194697c-4cc1-40b3-a497-8ef64c4f0ecd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.634269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194697c-4cc1-40b3-a497-8ef64c4f0ecd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.634555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194697c-4cc1-40b3-a497-8ef64c4f0ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.640793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=67 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:08:27.654530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4194697c-4cc1-40b3-a497-8ef64c4f0ecd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:27.656199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4194697c-4cc1-40b3-a497-8ef64c4f0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:27.665150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 714/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:08:27.713419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691e6b6-397b-4402-9fdc-580edfd4e865 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:27.729814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8691e6b6-397b-4402-9fdc-580edfd4e865 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:27.735506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:27.738061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:27.738061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:27.738061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:27.746824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:27.748558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd6c7d-d918-4849-b8db-da65d3e992eb service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:27.753861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:08:27.762806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 695/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:37.649348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:08:49.548549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764cc30-85da-4083-920f-7e3d4ae84b63 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:49.549251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764cc30-85da-4083-920f-7e3d4ae84b63 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:49.549251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764cc30-85da-4083-920f-7e3d4ae84b63 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:49.550988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 715/1410] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jun 30 04:08:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:49.557578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02be47-842c-46ce-be39-28b9e7a56703 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:49.558021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02be47-842c-46ce-be39-28b9e7a56703 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:49.558126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02be47-842c-46ce-be39-28b9e7a56703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:49.578349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ff02be47-842c-46ce-be39-28b9e7a56703 None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:49.579644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ff02be47-842c-46ce-be39-28b9e7a56703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:49.589912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 696/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 04:08:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:49.605035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6134-219f-4817-a8f7-a1a795152781 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:49.605332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6134-219f-4817-a8f7-a1a795152781 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:49.605644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d6134-219f-4817-a8f7-a1a795152781 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:49.607031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 716/1412] 10.222.0.13 () {54 vars in 918 bytes} [Thu Jun 30 04:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:49.611716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5694c29-107f-4423-98c6-37fa5a58a831 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:49.612012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5694c29-107f-4423-98c6-37fa5a58a831 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:49.612293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5694c29-107f-4423-98c6-37fa5a58a831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:49.633100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e5694c29-107f-4423-98c6-37fa5a58a831 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:49.634305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e5694c29-107f-4423-98c6-37fa5a58a831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:49.644468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 697/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 30 04:08:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:52.155577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086b27-87da-4c99-9f35-31fd48031a0d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:52.155577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086b27-87da-4c99-9f35-31fd48031a0d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:52.155577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2086b27-87da-4c99-9f35-31fd48031a0d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:52.155577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 04:08:52.160976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4eb92-464d-4153-ae2f-1e9b4505a06f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:52.161705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4eb92-464d-4153-ae2f-1e9b4505a06f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:52.161839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4eb92-464d-4153-ae2f-1e9b4505a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:52.182703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-aaa4eb92-464d-4153-ae2f-1e9b4505a06f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:52.183993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-aaa4eb92-464d-4153-ae2f-1e9b4505a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:52.193080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:52.253134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086b27-87da-4c99-9f35-31fd48031a0d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:52.276028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2086b27-87da-4c99-9f35-31fd48031a0d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:52.282591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:52.283305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:52.283593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:52.283871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:52.294500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:52.296156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f533f98-cf74-444f-9a0a-0e0cef809375 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:52.306492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 718/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:53.204213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f696d-65ec-471b-a4ce-67b2916c9351 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:53.204819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f696d-65ec-471b-a4ce-67b2916c9351 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:53.204819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f696d-65ec-471b-a4ce-67b2916c9351 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:53.206174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:53.212452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc47e1-e054-497c-b694-66c79e71010d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:53.213056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc47e1-e054-497c-b694-66c79e71010d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:53.213311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc47e1-e054-497c-b694-66c79e71010d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:53.234543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8ebc47e1-e054-497c-b694-66c79e71010d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:53.235856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8ebc47e1-e054-497c-b694-66c79e71010d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:53.245913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:53.395728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f696d-65ec-471b-a4ce-67b2916c9351 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:53.415537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f696d-65ec-471b-a4ce-67b2916c9351 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:53.422834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:53.423521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:53.423804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:53.424082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:53.432952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:53.434544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b570d-20c8-414a-add3-70e6312186c5 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:53.444385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 700/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:54.344841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898cf89-4f36-4686-a743-2b5027a776f3 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:54.345389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898cf89-4f36-4686-a743-2b5027a776f3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:54.345389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898cf89-4f36-4686-a743-2b5027a776f3 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:54.346793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:54.353408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4052b-a143-4460-9a2b-c7b2ae36ab37 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:54.353834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4052b-a143-4460-9a2b-c7b2ae36ab37 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:54.353990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4052b-a143-4460-9a2b-c7b2ae36ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:54.376936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1db4052b-a143-4460-9a2b-c7b2ae36ab37 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:54.378141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1db4052b-a143-4460-9a2b-c7b2ae36ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:54.386979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:54.551905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898cf89-4f36-4686-a743-2b5027a776f3 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:54.567809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898cf89-4f36-4686-a743-2b5027a776f3 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:54.574305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:54.575045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:54.575426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:54.575783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:54.587859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:54.587859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a54c4-0f8c-48af-84ed-0eb75674b4ab service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:54.597402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 721/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:55.449843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88f7fc-5da8-48dd-ae7d-3c89659b788a None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:55.450762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88f7fc-5da8-48dd-ae7d-3c89659b788a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:55.450762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88f7fc-5da8-48dd-ae7d-3c89659b788a None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:55.453108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:55.458255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9fe2-0e2e-4ee8-9c43-e41941562584 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:55.458719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9fe2-0e2e-4ee8-9c43-e41941562584 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:55.458830 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9fe2-0e2e-4ee8-9c43-e41941562584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:55.479068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-292e9fe2-0e2e-4ee8-9c43-e41941562584 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:55.480285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-292e9fe2-0e2e-4ee8-9c43-e41941562584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:55.488905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:55.637359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88f7fc-5da8-48dd-ae7d-3c89659b788a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:55.662718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88f7fc-5da8-48dd-ae7d-3c89659b788a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:55.668902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:55.669676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:55.669967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:55.670247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:55.679360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:55.680977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c01211-811b-4b4f-9d8a-d6c10e915476 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:55.691033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 703/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:56.568842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096387-406c-4f89-9924-3b41ea530d2b None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:56.568842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096387-406c-4f89-9924-3b41ea530d2b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:56.569513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23096387-406c-4f89-9924-3b41ea530d2b None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:56.570886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:56.576606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c120fe-b932-4bc7-bae5-efd12579d523 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:56.577075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c120fe-b932-4bc7-bae5-efd12579d523 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:56.577316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c120fe-b932-4bc7-bae5-efd12579d523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:56.601700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a2c120fe-b932-4bc7-bae5-efd12579d523 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:56.602944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a2c120fe-b932-4bc7-bae5-efd12579d523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:56.612432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:56.764167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23096387-406c-4f89-9924-3b41ea530d2b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:56.784039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23096387-406c-4f89-9924-3b41ea530d2b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:56.791313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:56.791968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:56.792258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:56.792536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:56.801324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:56.803041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d81392-8cef-4d77-b816-acb54a7ef65f service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:56.813827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 724/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:57.637935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694cd1-2434-456d-ad3e-185aa29a1a5d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:57.638538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694cd1-2434-456d-ad3e-185aa29a1a5d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:57.638538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af694cd1-2434-456d-ad3e-185aa29a1a5d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:57.639936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:57.645910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d39e88-7952-4d41-8800-28a802744169 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:57.646771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d39e88-7952-4d41-8800-28a802744169 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:57.646987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d39e88-7952-4d41-8800-28a802744169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:57.667037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f5d39e88-7952-4d41-8800-28a802744169 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:57.668298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f5d39e88-7952-4d41-8800-28a802744169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:57.678153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:57.837967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af694cd1-2434-456d-ad3e-185aa29a1a5d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:57.854080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af694cd1-2434-456d-ad3e-185aa29a1a5d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:57.859816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:57.860460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:57.860743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:57.861016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:57.870603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:57.872233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67a453-b22b-421f-bd58-efe66fd6f107 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:57.883130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 706/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:58.737988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fdd06-47a1-4ad4-824e-e3b0ea4ed9ff None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:58.738603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fdd06-47a1-4ad4-824e-e3b0ea4ed9ff None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:58.738603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7fdd06-47a1-4ad4-824e-e3b0ea4ed9ff None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:58.739967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:58.746104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09313e7-f387-4213-9dce-eb81a1a26066 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:58.746561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09313e7-f387-4213-9dce-eb81a1a26066 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:58.746683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09313e7-f387-4213-9dce-eb81a1a26066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:58.769922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b09313e7-f387-4213-9dce-eb81a1a26066 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:58.771161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b09313e7-f387-4213-9dce-eb81a1a26066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:58.781073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:58.937573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fdd06-47a1-4ad4-824e-e3b0ea4ed9ff None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:08:58.956180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7fdd06-47a1-4ad4-824e-e3b0ea4ed9ff None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:08:58.962597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:08:58.963277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:58.963565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:58.963845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:58.972515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:08:58.974107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fac1a4-5c41-4fc6-9043-43e082d77a0b service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:08:58.984222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 727/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:08:59.800618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b492-26d5-4f82-b90e-f9623fe8af03 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:59.801184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b492-26d5-4f82-b90e-f9623fe8af03 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:59.801184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b492-26d5-4f82-b90e-f9623fe8af03 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:59.802827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:08:59.810129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969331b9-90e1-4d52-bf51-aecb1a25d8dc None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:08:59.810569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969331b9-90e1-4d52-bf51-aecb1a25d8dc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:08:59.810752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969331b9-90e1-4d52-bf51-aecb1a25d8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:08:59.838665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-969331b9-90e1-4d52-bf51-aecb1a25d8dc None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:08:59.839854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-969331b9-90e1-4d52-bf51-aecb1a25d8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:08:59.848289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:08:59.999819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b492-26d5-4f82-b90e-f9623fe8af03 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:00.024490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b492-26d5-4f82-b90e-f9623fe8af03 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:00.031461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:00.032151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:00.032458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:00.032715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:00.041435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:00.043682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19741a51-3997-4367-ad6b-d2d63cfa3716 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:00.054226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 709/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:00.869492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32336-d1dc-4baa-9a0d-249caf2bcfdc None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:00.870405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32336-d1dc-4baa-9a0d-249caf2bcfdc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:00.870405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32336-d1dc-4baa-9a0d-249caf2bcfdc None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:00.872448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:09:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:00.878603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abec7-e3fb-44bd-b22a-22c2cfb37a92 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:00.879063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abec7-e3fb-44bd-b22a-22c2cfb37a92 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:00.879519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9abec7-e3fb-44bd-b22a-22c2cfb37a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:00.899289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7f9abec7-e3fb-44bd-b22a-22c2cfb37a92 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:00.900477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7f9abec7-e3fb-44bd-b22a-22c2cfb37a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:00.908809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:01.076480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc32336-d1dc-4baa-9a0d-249caf2bcfdc None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:01.093240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc32336-d1dc-4baa-9a0d-249caf2bcfdc None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:01.100349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:01.100991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:01.101278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:01.101552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:01.110100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:01.111726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f01592-cade-47df-8ee3-55e9322f09e1 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:01.122026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 730/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:03.031875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321d850-9e8b-4b17-8e9c-7cde48f7a538 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:03.032519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321d850-9e8b-4b17-8e9c-7cde48f7a538 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:03.032519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321d850-9e8b-4b17-8e9c-7cde48f7a538 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:03.033816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:03.041836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16607f-f803-439e-ae32-e949340e5e93 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:03.042729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16607f-f803-439e-ae32-e949340e5e93 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:03.043734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16607f-f803-439e-ae32-e949340e5e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:03.071512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6f16607f-f803-439e-ae32-e949340e5e93 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:03.072765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6f16607f-f803-439e-ae32-e949340e5e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:03.084838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:03.127071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321d850-9e8b-4b17-8e9c-7cde48f7a538 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:03.140501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321d850-9e8b-4b17-8e9c-7cde48f7a538 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:03.144341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:03.144984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:03.145269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:03.145548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:03.155664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:03.157112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc442207-e74e-49cb-83f2-d02475f62960 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:03.166883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 712/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:05.345483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa02af-a163-4941-9418-f43b6054b148 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:05.346346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa02af-a163-4941-9418-f43b6054b148 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:05.346346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa02af-a163-4941-9418-f43b6054b148 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:05.348404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:05.354009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d2c-4854-421d-a1ee-0060622587b0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:05.354451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d2c-4854-421d-a1ee-0060622587b0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:05.354556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf1d2c-4854-421d-a1ee-0060622587b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:05.373773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-93bf1d2c-4854-421d-a1ee-0060622587b0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:05.374972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-93bf1d2c-4854-421d-a1ee-0060622587b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:05.383375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:05.425489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa02af-a163-4941-9418-f43b6054b148 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:05.444802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa02af-a163-4941-9418-f43b6054b148 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:05.448853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:05.449510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:05.449795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:05.450075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:05.460589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:05.462061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92040ff-ce4a-405a-8230-fe5580867ae7 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:05.472155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 733/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:07.906993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb795b-4d2b-4b47-ad45-aa0d9ed382a1 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:07.907942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb795b-4d2b-4b47-ad45-aa0d9ed382a1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:07.907942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb795b-4d2b-4b47-ad45-aa0d9ed382a1 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:07.909900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:07.916953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3b71-543a-41b5-a141-9076bfc4d26c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:07.917399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3b71-543a-41b5-a141-9076bfc4d26c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:07.917504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3b71-543a-41b5-a141-9076bfc4d26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:07.938898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f78a3b71-543a-41b5-a141-9076bfc4d26c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:07.940148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f78a3b71-543a-41b5-a141-9076bfc4d26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:07.949848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:08.000577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb795b-4d2b-4b47-ad45-aa0d9ed382a1 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:08.019080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb795b-4d2b-4b47-ad45-aa0d9ed382a1 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:08.024278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:08.024917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:08.025206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:08.025477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:08.035278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:08.036722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471166c3-6e8a-494b-9262-893974d4447d service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:08.047713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 715/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:10.226689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54c-27e6-4647-a266-f536c9dc4e88 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:10.227259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54c-27e6-4647-a266-f536c9dc4e88 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:10.227361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54c-27e6-4647-a266-f536c9dc4e88 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:10.228778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:10.234143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b5991-e15b-4b1c-8ef7-ab851e256c09 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:10.234590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b5991-e15b-4b1c-8ef7-ab851e256c09 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:10.234686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b5991-e15b-4b1c-8ef7-ab851e256c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:10.253287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2c6b5991-e15b-4b1c-8ef7-ab851e256c09 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:10.254493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2c6b5991-e15b-4b1c-8ef7-ab851e256c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:10.262974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:10.301527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e54c-27e6-4647-a266-f536c9dc4e88 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:10.313329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e54c-27e6-4647-a266-f536c9dc4e88 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:10.317067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:10.317712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:10.317997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:10.318308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:10.327681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:10.329109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a69421-cf83-49ea-81f0-9564af28c8d3 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:10.338991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 736/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:09:12.458140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd79fa-fa6b-4ab8-b437-c3356976888f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:12.458689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd79fa-fa6b-4ab8-b437-c3356976888f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:12.458689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd79fa-fa6b-4ab8-b437-c3356976888f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:12.460205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:12.466671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2ac1-4f4c-4587-a169-9fbe8d7fb948 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:12.467361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2ac1-4f4c-4587-a169-9fbe8d7fb948 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:12.467610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed2ac1-4f4c-4587-a169-9fbe8d7fb948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:12.495465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-11ed2ac1-4f4c-4587-a169-9fbe8d7fb948 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:12.496693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-11ed2ac1-4f4c-4587-a169-9fbe8d7fb948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:12.507031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:12.550299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd79fa-fa6b-4ab8-b437-c3356976888f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:12.562028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd79fa-fa6b-4ab8-b437-c3356976888f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:12.565999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:12.566649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:12.566936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:12.567266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:12.577402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:12.578856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d80a5-73fe-4701-9ec9-1f3ab862ad8a service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:12.589182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 718/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:14.735348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99faf50-554c-40d4-a110-d04f817d30f2 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:14.736308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99faf50-554c-40d4-a110-d04f817d30f2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:14.736308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99faf50-554c-40d4-a110-d04f817d30f2 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:14.738412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:14.743984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a162329-4ce1-4c41-98d4-052cc3200e50 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:14.744452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a162329-4ce1-4c41-98d4-052cc3200e50 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:14.744554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a162329-4ce1-4c41-98d4-052cc3200e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:14.763205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9a162329-4ce1-4c41-98d4-052cc3200e50 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:14.764423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9a162329-4ce1-4c41-98d4-052cc3200e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:14.774651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:14.817255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99faf50-554c-40d4-a110-d04f817d30f2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:14.835767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99faf50-554c-40d4-a110-d04f817d30f2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:14.840945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:14.841842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:14.842122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:14.842488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:14.854434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:14.855914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae987e-637c-4e56-80fb-29420a215b3c service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:14.866040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 739/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:17.036924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e9d3-42bc-44b2-9ea2-d98f2a290551 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:17.037521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e9d3-42bc-44b2-9ea2-d98f2a290551 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:17.037521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e9d3-42bc-44b2-9ea2-d98f2a290551 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:17.039059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:17.044923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481f63-d73d-4e0c-bea0-23931799d2cc None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:17.045362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481f63-d73d-4e0c-bea0-23931799d2cc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:17.045463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d481f63-d73d-4e0c-bea0-23931799d2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:17.070275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4d481f63-d73d-4e0c-bea0-23931799d2cc None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:17.072130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4d481f63-d73d-4e0c-bea0-23931799d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:17.086673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:17.140135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340e9d3-42bc-44b2-9ea2-d98f2a290551 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:17.156170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340e9d3-42bc-44b2-9ea2-d98f2a290551 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:17.162641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:17.163687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:17.164200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:17.164484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:17.180003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:17.181675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85241e4-cda4-4ae7-84eb-4e8efc329271 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:17.192103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 721/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:19.272946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052168-e307-4191-8ad8-1bb14d30a7ee None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:19.273961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052168-e307-4191-8ad8-1bb14d30a7ee None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:19.274249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052168-e307-4191-8ad8-1bb14d30a7ee None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:19.275717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:19.281096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd7d3-3beb-4ce3-8863-5a26442aceab None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:19.281546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd7d3-3beb-4ce3-8863-5a26442aceab None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:19.281669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd7d3-3beb-4ce3-8863-5a26442aceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:19.288089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=79 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:09:19.301880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-178cd7d3-3beb-4ce3-8863-5a26442aceab None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:19.303089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-178cd7d3-3beb-4ce3-8863-5a26442aceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:19.312109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:19.354281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052168-e307-4191-8ad8-1bb14d30a7ee None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:19.364008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=73 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:09:19.371203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052168-e307-4191-8ad8-1bb14d30a7ee None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:19.375456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:19.376102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:19.376385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:19.376655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:19.385939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:19.387411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d9419-0fa1-4fcb-9be5-e68989775234 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:19.397315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 742/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:21.461704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a79be-2b2c-4b3e-86e6-24e09b70cd1c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:21.462314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a79be-2b2c-4b3e-86e6-24e09b70cd1c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:21.462314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a79be-2b2c-4b3e-86e6-24e09b70cd1c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:21.463672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:21.469866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171ebe-7cac-44ea-9eef-33118374177e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:21.470530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171ebe-7cac-44ea-9eef-33118374177e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:21.470706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171ebe-7cac-44ea-9eef-33118374177e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:21.496914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3c171ebe-7cac-44ea-9eef-33118374177e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:21.498125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3c171ebe-7cac-44ea-9eef-33118374177e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:21.507274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:21.544166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a79be-2b2c-4b3e-86e6-24e09b70cd1c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:21.555745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a79be-2b2c-4b3e-86e6-24e09b70cd1c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:21.559729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:21.560370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:21.560657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:21.560936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:21.570531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:21.572032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd38a18-77b0-4168-9dff-f35bade13832 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:21.582152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 724/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:23.633910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232e2f4-9340-439d-8c57-887f85efda44 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:23.634717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232e2f4-9340-439d-8c57-887f85efda44 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:23.634914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2232e2f4-9340-439d-8c57-887f85efda44 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:23.637128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:09:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:23.642608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aa65c-8dbf-4222-bca9-81d2a67f38f8 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:23.643045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aa65c-8dbf-4222-bca9-81d2a67f38f8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:23.643179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486aa65c-8dbf-4222-bca9-81d2a67f38f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:23.670858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-486aa65c-8dbf-4222-bca9-81d2a67f38f8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:23.672104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-486aa65c-8dbf-4222-bca9-81d2a67f38f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:23.682934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:23.727994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232e2f4-9340-439d-8c57-887f85efda44 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:23.739824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2232e2f4-9340-439d-8c57-887f85efda44 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:23.744530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:23.745176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:23.745464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:23.745743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:23.755459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:23.756894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048afec-c4b2-41f1-b326-4ebd4efa2930 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:23.766450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 745/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:33.665105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=11 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:09:33.762491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=14 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:09:44.434089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace51dcd-be70-425c-93f2-88e750e084f5 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:44.434682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace51dcd-be70-425c-93f2-88e750e084f5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:44.434682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace51dcd-be70-425c-93f2-88e750e084f5 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:44.436055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 726/1471] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:44.442661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6694f68-e12b-434c-a0ba-ba0027fb517c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:44.443457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6694f68-e12b-434c-a0ba-ba0027fb517c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:44.443598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6694f68-e12b-434c-a0ba-ba0027fb517c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:44.463780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d6694f68-e12b-434c-a0ba-ba0027fb517c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:44.464992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d6694f68-e12b-434c-a0ba-ba0027fb517c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:44.475025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 746/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:44.536583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db5b6-04a6-48da-bb83-94ce179223a3 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:44.536882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db5b6-04a6-48da-bb83-94ce179223a3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:44.537159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db5b6-04a6-48da-bb83-94ce179223a3 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:44.538980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 727/1473] 10.222.0.13 () {54 vars in 962 bytes} [Thu Jun 30 04:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:44.544076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe801-1f7a-459a-9e3c-ff3ac9edbf45 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:44.544398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe801-1f7a-459a-9e3c-ff3ac9edbf45 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:44.544711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fe801-1f7a-459a-9e3c-ff3ac9edbf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:44.563577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-731fe801-1f7a-459a-9e3c-ff3ac9edbf45 None None] MFA Rules not processed for user `aecdfa0fc8514b09b04bd7ee759d21ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:44.564802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-731fe801-1f7a-459a-9e3c-ff3ac9edbf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:44.572870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 747/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Thu Jun 30 04:09:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:44.580358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db5b6-04a6-48da-bb83-94ce179223a3 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:44.593026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db5b6-04a6-48da-bb83-94ce179223a3 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:44.597458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:44.598114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:44.598401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:44.598674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:44.608772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:44.610223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c7bf1-98b3-42bd-8225-ec0e66ffc413 service glance] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:44.620450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 728/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:46.750015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b27d-b030-46fa-ba23-15d6f60b8622 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:46.750584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b27d-b030-46fa-ba23-15d6f60b8622 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:46.750584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103b27d-b030-46fa-ba23-15d6f60b8622 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:46.752003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 748/1476] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:46.758542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5edfe-df01-4e1f-a1b8-30f92f0a6472 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:46.759041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5edfe-df01-4e1f-a1b8-30f92f0a6472 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:46.759041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5edfe-df01-4e1f-a1b8-30f92f0a6472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:46.777696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f7c5edfe-df01-4e1f-a1b8-30f92f0a6472 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:46.778903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f7c5edfe-df01-4e1f-a1b8-30f92f0a6472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:46.789458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 729/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:46.825648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103b27d-b030-46fa-ba23-15d6f60b8622 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:46.846242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103b27d-b030-46fa-ba23-15d6f60b8622 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:46.853182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:46.853847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:46.854134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:46.854415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:46.855189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:46.856993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:09:46.856993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:09:46.856993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:09:46.858142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eebd372-3903-4b2f-b2b7-9dea4f7a90f2 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:46.877985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 749/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Thu Jun 30 04:09:46 2022] GET /identity/v3/projects => generated 2553 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:09:48.926252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6d1e-1530-4c50-9658-8136b686283d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:48.927127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6d1e-1530-4c50-9658-8136b686283d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:48.927127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6d1e-1530-4c50-9658-8136b686283d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:48.929843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 730/1479] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:48.935348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657193-4254-4bc0-b613-c9d4781fe3f0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:48.935807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657193-4254-4bc0-b613-c9d4781fe3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:48.935913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48657193-4254-4bc0-b613-c9d4781fe3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:48.955480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-48657193-4254-4bc0-b613-c9d4781fe3f0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:48.956689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-48657193-4254-4bc0-b613-c9d4781fe3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:48.965253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 750/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:49.008832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f6d1e-1530-4c50-9658-8136b686283d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:49.034160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f6d1e-1530-4c50-9658-8136b686283d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:49.042459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:49.043590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:49.043875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:49.044150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:49.052748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:49.054355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffe744-aa2a-40dd-8271-7e9138b56eaf service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:49.064016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 731/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:51.292118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56eaab-0cce-4467-bc03-fa6cf5960141 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:51.293072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56eaab-0cce-4467-bc03-fa6cf5960141 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:51.293072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56eaab-0cce-4467-bc03-fa6cf5960141 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:51.294402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 751/1482] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:51.300051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601353c2-95be-4405-b459-d532f08f55af None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:51.300519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601353c2-95be-4405-b459-d532f08f55af None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:51.300610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601353c2-95be-4405-b459-d532f08f55af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:51.320847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-601353c2-95be-4405-b459-d532f08f55af None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:51.322074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-601353c2-95be-4405-b459-d532f08f55af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:51.331438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 732/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:51.367989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56eaab-0cce-4467-bc03-fa6cf5960141 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:51.379042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56eaab-0cce-4467-bc03-fa6cf5960141 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:51.382820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:51.383570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:51.383859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:51.384136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:51.393409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:51.394871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc98c4-34e3-4ad5-b640-68bdb165dbd5 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:51.404675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 752/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:53.490721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb8ee-454f-477f-9740-2b27910ddd74 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:53.491653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb8ee-454f-477f-9740-2b27910ddd74 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:53.491653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afb8ee-454f-477f-9740-2b27910ddd74 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:53.492843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 733/1485] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:53.499328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e917-e95a-4862-9b27-8a3025952ec7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:53.500024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e917-e95a-4862-9b27-8a3025952ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:53.500210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e917-e95a-4862-9b27-8a3025952ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:53.521630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e719e917-e95a-4862-9b27-8a3025952ec7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:53.522891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e719e917-e95a-4862-9b27-8a3025952ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:53.533261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 753/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:53.574149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb8ee-454f-477f-9740-2b27910ddd74 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:53.586292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afb8ee-454f-477f-9740-2b27910ddd74 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:53.590395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:53.591040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:53.591341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:53.591675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:53.601711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:53.603170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd139148-2382-470b-91a0-51f6a6fd10c6 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:53.612700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 734/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:55.758462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dd86-d3ca-4502-9e72-b9d49a12954c None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:55.759010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dd86-d3ca-4502-9e72-b9d49a12954c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:55.759010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dd86-d3ca-4502-9e72-b9d49a12954c None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:55.760426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 754/1488] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:55.766976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4bb0-50e1-4aa2-b418-07dbea8d8d88 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:55.767833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4bb0-50e1-4aa2-b418-07dbea8d8d88 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:55.768000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4bb0-50e1-4aa2-b418-07dbea8d8d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:55.794609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e96e4bb0-50e1-4aa2-b418-07dbea8d8d88 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:55.795866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e96e4bb0-50e1-4aa2-b418-07dbea8d8d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:55.804933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 735/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:55.848290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0dd86-d3ca-4502-9e72-b9d49a12954c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:55.861032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0dd86-d3ca-4502-9e72-b9d49a12954c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:55.865337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:55.865979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:55.866270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:55.866696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:55.876548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:55.877993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e8f4f-85cb-479b-9a89-893b084661fe service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:55.887686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 755/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:09:58.164709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698e9-a0a3-4465-847f-b746fec4185d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:58.165242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698e9-a0a3-4465-847f-b746fec4185d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:58.165242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788698e9-a0a3-4465-847f-b746fec4185d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:58.167296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 736/1491] 10.222.0.13 () {54 vars in 926 bytes} [Thu Jun 30 04:09:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:09:58.174790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c81d-c49b-4dac-96c7-4d221c98401a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:58.175311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c81d-c49b-4dac-96c7-4d221c98401a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:58.175628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c81d-c49b-4dac-96c7-4d221c98401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:58.201653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2708c81d-c49b-4dac-96c7-4d221c98401a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:09:58.202905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2708c81d-c49b-4dac-96c7-4d221c98401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:09:58.212327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 756/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Thu Jun 30 04:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:09:58.260480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788698e9-a0a3-4465-847f-b746fec4185d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:09:58.272296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788698e9-a0a3-4465-847f-b746fec4185d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:09:58.276415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:09:58.277073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:09:58.277349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:09:58.277625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:09:58.287364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:09:58.289023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7baa22-7133-4a2c-a848-9bbb5d6a10cc service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:09:58.299612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 737/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:10:08.196134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=33 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:10:08.294828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=33 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:11:25.069799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259162d-863c-4209-8265-ced483993f9a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.070423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259162d-863c-4209-8265-ced483993f9a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.070423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259162d-863c-4209-8265-ced483993f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.091595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8259162d-863c-4209-8265-ced483993f9a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:11:25.092813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8259162d-863c-4209-8265-ced483993f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:25.102247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 757/1494] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:11:25.110518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.111032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.111032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.129294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:11:25.130490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:25.139587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 738/1495] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:11:25.144206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259162d-863c-4209-8265-ced483993f9a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.155379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259162d-863c-4209-8265-ced483993f9a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.161641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.162301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.162581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.162852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.164265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:11:25.164577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.165606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.165606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:11:25.165606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:11:25.166875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.178559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 758/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:11:25.190527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.201392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601ec1d-f5a0-4541-aff2-06aeb5093bf2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.206703 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.207395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.207679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.207951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.208638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.211009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.211009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:11:25.211009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:11:25.212244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.252099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 739/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:11:25.256374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.270586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dd345-8a76-4e63-8b1e-a430c7fe7ebc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.337694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.338336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.338649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.338923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.339686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.340707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.340707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:11:25.340707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:11:25.341916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.362767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 759/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:11:25.367577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.377879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205a471-23af-4cd9-9d4c-919ff9900778 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.382978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.383666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.383949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.384223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.384955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.385976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.385976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:11:25.385976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:11:25.386850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.400134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 740/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:11:25.405686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.419584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197aaba4-e0c7-451f-ad62-222b4848a806 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.426944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.427700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.427984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.428253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.429232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.430276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.430276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:11:25.430276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:11:25.431178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de945d7a-ad64-4e82-a656-e204a90d5cb1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.444916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 760/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:11:25.449355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.459470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a78e0-a682-4783-9dbe-ff5429cafe2e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.466270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.467216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.467503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.467783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] PATH_INFO: `/v3/projects/deaf2561aa484bcaac2738ced5c45171/users/55569a4fa5e543be85af80332901a497/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.483762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deaf2561aa484bcaac2738ced5c45171, user_id=55569a4fa5e543be85af80332901a497, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.484818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.484818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:11:25.484818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:11:25.486758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f098a-b392-47d4-b723-fa0e7c909bc3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.502005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 741/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:11:25 2022] PUT /identity/v3/projects/deaf2561aa484bcaac2738ced5c45171/users/55569a4fa5e543be85af80332901a497/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:11:25.514276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ec10-60c5-426f-96ba-1bca97398571 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.514575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ec10-60c5-426f-96ba-1bca97398571 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.514854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3ec10-60c5-426f-96ba-1bca97398571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.541983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-12d3ec10-60c5-426f-96ba-1bca97398571 None None] MFA Rules not processed for user `55569a4fa5e543be85af80332901a497`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:11:25.543222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-12d3ec10-60c5-426f-96ba-1bca97398571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:25.611527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 761/1502] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:11:25.624992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6bdf-a616-4570-ac43-5418d4198ef2 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.625291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6bdf-a616-4570-ac43-5418d4198ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.625569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb6bdf-a616-4570-ac43-5418d4198ef2 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.626974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 742/1503] 10.222.0.13 () {54 vars in 963 bytes} [Thu Jun 30 04:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:11:25.631873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f57d-88f4-4140-9e05-543017d85cb2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.632171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f57d-88f4-4140-9e05-543017d85cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.632451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59f57d-88f4-4140-9e05-543017d85cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.650689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7e59f57d-88f4-4140-9e05-543017d85cb2 None None] MFA Rules not processed for user `d534d56b90834341afb915c39e70faff`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:11:25.651911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7e59f57d-88f4-4140-9e05-543017d85cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:25.699285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:11:25.708456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 762/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Thu Jun 30 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:11:25.720166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb6bdf-a616-4570-ac43-5418d4198ef2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.733188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb6bdf-a616-4570-ac43-5418d4198ef2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.737175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.737819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.738107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.738381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.747370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.748810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.757778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 743/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:11:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:11:25.801578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74bb51-5a4c-4ecc-990a-9fc32a7191c3 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.801880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74bb51-5a4c-4ecc-990a-9fc32a7191c3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.802160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74bb51-5a4c-4ecc-990a-9fc32a7191c3 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.803519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 763/1506] 10.222.0.13 () {52 vars in 829 bytes} [Thu Jun 30 04:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:11:25.807912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:25.874359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c548691-4f82-41d8-a600-91774a3bce74 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:25.882666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:25.883596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:25.883915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:25.884194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] PATH_INFO: `/v3/users/55569a4fa5e543be85af80332901a497` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:25.885488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=55569a4fa5e543be85af80332901a497)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:25.886533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:25.886533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:11:25.886533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:11:25.887817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:25.966547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55569a4fa5e543be85af80332901a497'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:25.982257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55569a4fa5e543be85af80332901a497'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.000408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55569a4fa5e543be85af80332901a497'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.001143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55569a4fa5e543be85af80332901a497'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.005285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:11:26.014377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55569a4fa5e543be85af80332901a497'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.021578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 744/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:11:25 2022] DELETE /identity/v3/users/55569a4fa5e543be85af80332901a497 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:11:26.026321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:26.026674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:26.026964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:26.044985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:11:26.046215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:26.117630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 764/1508] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:11:26.128139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:26.129632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:26.200675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767867bc-30d7-4b49-9738-d00ec63b36ef admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:26.205085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:26.205752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:26.206039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:26.206319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:26.216170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:26.217633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8be24-2a0b-4404-8ce1-be30a437a9f1 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:26.227792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 745/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:11:26.575924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:11:26.577474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:11:26.594196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bab830-d083-49c2-b330-02b014a61a25 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:11:26.604320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:11:26.605215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:11:26.605503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:11:26.605819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] PATH_INFO: `/v3/projects/deaf2561aa484bcaac2738ced5c45171` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:11:26.607033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=deaf2561aa484bcaac2738ced5c45171)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:11:26.608108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:11:26.608108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:11:26.608108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:11:26.609296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:11:26.673047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deaf2561aa484bcaac2738ced5c45171', 'request_id': 'req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.684493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deaf2561aa484bcaac2738ced5c45171', 'request_id': 'req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.690658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deaf2561aa484bcaac2738ced5c45171', 'request_id': 'req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.710127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] The token cache is being invalidate because project deaf2561aa484bcaac2738ced5c45171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:11:26.710542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.711300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-60cb68a7-4283-4774-9e2c-e6465dd4cb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:11:26.713129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 765/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:11:26 2022] DELETE /identity/v3/projects/deaf2561aa484bcaac2738ced5c45171 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:11:36.223670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:11:36.706668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:13:04.573896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:04.574461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:04.574461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:04.575872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:04.581251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88714bf6-4fcc-4942-8f1a-e97f37e38654 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:04.582162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88714bf6-4fcc-4942-8f1a-e97f37e38654 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:04.582518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88714bf6-4fcc-4942-8f1a-e97f37e38654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:04.603117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-88714bf6-4fcc-4942-8f1a-e97f37e38654 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:04.604348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-88714bf6-4fcc-4942-8f1a-e97f37e38654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:04.671295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:04.840870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:04.843941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:04.939224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fd0b3-a375-4a08-9ae9-389db30e25ed None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:04.945429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:04.946089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:04.946386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:04.946661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:04.955124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:04.956904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8e79cc-48ee-48a4-b342-d77dd443eccf service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:04.967216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 747/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:05.813918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a112-09a3-461e-9c77-b586643e1c08 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:05.814518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a112-09a3-461e-9c77-b586643e1c08 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:05.814518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5a112-09a3-461e-9c77-b586643e1c08 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:05.816414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:05.822485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74d1-771c-4113-a06e-cc5eb9f45103 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:05.823260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74d1-771c-4113-a06e-cc5eb9f45103 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:05.823428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae74d1-771c-4113-a06e-cc5eb9f45103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:05.845721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5bae74d1-771c-4113-a06e-cc5eb9f45103 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:05.847298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5bae74d1-771c-4113-a06e-cc5eb9f45103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:05.856653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:06.016800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a112-09a3-461e-9c77-b586643e1c08 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:06.039763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5a112-09a3-461e-9c77-b586643e1c08 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:06.047371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:06.048017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:06.048302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:06.048645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:06.057513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:06.059126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e00443d-0572-4ae1-b4c0-6de09c29e720 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:06.069176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 768/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:06.630181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c68c58-2366-4625-b547-139d26370ff3 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:06.630473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c68c58-2366-4625-b547-139d26370ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:06.630806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c68c58-2366-4625-b547-139d26370ff3 None None] PATH_INFO: `/v3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:06.632190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 749/1517] 10.222.0.20 () {54 vars in 925 bytes} [Thu Jun 30 04:13:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:13:06.638900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb601b-d597-4eb4-9c4c-01264b937614 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:06.639219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb601b-d597-4eb4-9c4c-01264b937614 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:06.639513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb601b-d597-4eb4-9c4c-01264b937614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:06.659612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0ecb601b-d597-4eb4-9c4c-01264b937614 None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:06.660986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0ecb601b-d597-4eb4-9c4c-01264b937614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:06.727380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 769/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jun 30 04:13:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:10.595059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19f69-6e4d-4c70-aa7e-aa8b0c44a613 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:10.595622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19f69-6e4d-4c70-aa7e-aa8b0c44a613 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:10.595622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e19f69-6e4d-4c70-aa7e-aa8b0c44a613 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:10.597030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 750/1519] 10.222.0.13 () {54 vars in 960 bytes} [Thu Jun 30 04:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:10.602217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d00ab-a14f-450d-ac12-7e6f07597c4e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:10.602685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d00ab-a14f-450d-ac12-7e6f07597c4e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:10.602788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d00ab-a14f-450d-ac12-7e6f07597c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:10.623364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c78d00ab-a14f-450d-ac12-7e6f07597c4e None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:10.624616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c78d00ab-a14f-450d-ac12-7e6f07597c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:10.634714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 770/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Thu Jun 30 04:13:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:10.642125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e19f69-6e4d-4c70-aa7e-aa8b0c44a613 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:10.656560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e19f69-6e4d-4c70-aa7e-aa8b0c44a613 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:10.662033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:10.662643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:10.662927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:10.663234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:10.672102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:10.673686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1b7c0-6d0d-49d8-b3ef-7b1ac1690196 service placement] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:10.683617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 751/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:17.323857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac718a2-fced-4dc3-8177-0ebc908ab92a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:17.324472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac718a2-fced-4dc3-8177-0ebc908ab92a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:17.324472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac718a2-fced-4dc3-8177-0ebc908ab92a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:17.326547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:13:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:17.334441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d961-e548-4252-84c3-64ba8600e88b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:17.334852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d961-e548-4252-84c3-64ba8600e88b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:17.334999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d961-e548-4252-84c3-64ba8600e88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:17.356651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0d45d961-e548-4252-84c3-64ba8600e88b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:17.357870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0d45d961-e548-4252-84c3-64ba8600e88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:17.368146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:17.536497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac718a2-fced-4dc3-8177-0ebc908ab92a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:17.551776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac718a2-fced-4dc3-8177-0ebc908ab92a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:17.558181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:17.558840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:17.559125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:17.559438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:17.568968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:17.570638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86350d9f-0ce7-46c9-a81c-45ca216e7964 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:17.580665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 772/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:18.402595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dce2d6-904e-4f33-9b87-13c9cbda4007 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:18.403204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dce2d6-904e-4f33-9b87-13c9cbda4007 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:18.403204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dce2d6-904e-4f33-9b87-13c9cbda4007 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:18.406213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 04:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:18.410705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac32c1-27e8-41e3-922b-cfa676e5d64b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:18.411458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac32c1-27e8-41e3-922b-cfa676e5d64b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:18.412429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac32c1-27e8-41e3-922b-cfa676e5d64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:18.438634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-16ac32c1-27e8-41e3-922b-cfa676e5d64b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:18.440716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-16ac32c1-27e8-41e3-922b-cfa676e5d64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:18.451878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 04:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:18.603537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dce2d6-904e-4f33-9b87-13c9cbda4007 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:18.620443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dce2d6-904e-4f33-9b87-13c9cbda4007 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:18.626481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:18.627142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:18.627465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:18.627746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:18.636473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:18.638036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0b252-df2c-4f1c-8408-82cf898c6693 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:18.647948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 754/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:22.692352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:22.693226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:22.693436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:22.694896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:22.700637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da55a8-dd38-4b8b-8ff4-e3577ed0a9b3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:22.701096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da55a8-dd38-4b8b-8ff4-e3577ed0a9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:22.701197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da55a8-dd38-4b8b-8ff4-e3577ed0a9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:22.721896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f1da55a8-dd38-4b8b-8ff4-e3577ed0a9b3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:22.723134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f1da55a8-dd38-4b8b-8ff4-e3577ed0a9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:22.732931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:13:22.769257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:22.772107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:22.835961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14739c-93e5-4465-bb50-6809b9d0841a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:22.839732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:22.840395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:22.840686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:22.840965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:22.850537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:22.851991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82283d6-8dec-471a-a222-379883a5497d service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:22.856913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:13:22.862079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 775/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:24.910354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:24.911238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:24.911238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:24.912958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:13:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:24.919812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422848d-54c4-454c-b945-879977084d1b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:24.920245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422848d-54c4-454c-b945-879977084d1b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:24.920380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422848d-54c4-454c-b945-879977084d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:24.946482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1422848d-54c4-454c-b945-879977084d1b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:24.947838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1422848d-54c4-454c-b945-879977084d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:24.956208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:24.993210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:24.996349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:25.013738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d0b52-35e3-4581-bec8-f049d04f3f2a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:25.017533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:25.018187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:25.018475 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:25.018766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:25.028975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:25.030421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091f1b9-f254-494e-8870-d7ccce68cae3 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:25.040755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 757/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:28.474173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:28.474872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:28.474872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:28.476303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:28.481686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887a48-8a79-44d3-b602-6e3bec4a6d73 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:28.482142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887a48-8a79-44d3-b602-6e3bec4a6d73 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:28.482246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea887a48-8a79-44d3-b602-6e3bec4a6d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:28.488602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=62 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:13:28.501620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ea887a48-8a79-44d3-b602-6e3bec4a6d73 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:28.502824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ea887a48-8a79-44d3-b602-6e3bec4a6d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:28.510747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:28.816142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:28.818128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:28.877797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010c6a-3b62-494c-bfcb-2e0f538621ca None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:28.881847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:28.882495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:28.882781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:28.883053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:28.892438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:28.893870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0739b94-14ad-4eb7-9d73-7f5d22ca8d6d service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:28.903530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 778/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:30.967274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:30.968095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:30.968095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:30.970374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 04:13:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:13:30.977790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed854e-ed42-471a-a6dc-e07e6f165096 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:30.978213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed854e-ed42-471a-a6dc-e07e6f165096 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:30.978354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed854e-ed42-471a-a6dc-e07e6f165096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:30.997241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2aed854e-ed42-471a-a6dc-e07e6f165096 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:13:30.998467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2aed854e-ed42-471a-a6dc-e07e6f165096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:31.007839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 04:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:13:31.308071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:13:31.310184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:13:31.328035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1db9cf-5a93-4cb3-878e-c004c7ffeb8f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:13:31.335363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:13:31.336144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:13:31.336428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:13:31.336796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:13:31.348944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:13:31.350780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dc44c-ca12-48b3-9189-e4662741fc3b service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:13:31.363061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 760/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:13:40.992133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=32 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:13:41.357817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=8 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:14:59.485437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.488410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.488410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.509275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.510863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:14:59.524006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 780/1540] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:14:59.533911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c261-da25-45ac-a81b-8e5073e5ac8d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.534918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c261-da25-45ac-a81b-8e5073e5ac8d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.535430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666c261-da25-45ac-a81b-8e5073e5ac8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.564182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0666c261-da25-45ac-a81b-8e5073e5ac8d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.565816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0666c261-da25-45ac-a81b-8e5073e5ac8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:14:59.582454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 761/1541] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:14:59.591155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:14:59.591414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18836-800e-4958-8cd0-0c2bb49514e9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.591414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18836-800e-4958-8cd0-0c2bb49514e9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.591414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce18836-800e-4958-8cd0-0c2bb49514e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.603722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d6e7-795e-4373-9502-6f2ef4f515a2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:14:59.614572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:14:59.614572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.614572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.614572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.615487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:14:59.615487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:14:59.615487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:14:59.628347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5ce18836-800e-4958-8cd0-0c2bb49514e9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.629598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5ce18836-800e-4958-8cd0-0c2bb49514e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:14:59.642478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 781/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:14:59.649005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 762/1543] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:14:59.650575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:14:59.657781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.658067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.658347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.669570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96755626-327d-4a64-8df2-7960e73d8fc7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:14:59.676389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:14:59.677076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.677363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.677645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.678359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:14:59.679413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:14:59.679413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:14:59.679413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:14:59.680617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:14:59.691674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.692894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:14:59.713334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 763/1544] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:14:59.721149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:14:59.733906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e21111-5387-445e-82b9-70ebd52e1283 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:14:59.742554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 782/1545] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:14:59.744187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:14:59.751832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:14:59.752526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.752817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.753094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.753836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:14:59.754147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:14:59.755868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:14:59.755868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:14:59.755868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:14:59.756355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:14:59.783237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 764/1546] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:14:59.785247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:14:59.801498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd849777-29df-4138-a4ad-5344cc8c2424 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:14:59.878468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd4343-70a1-43a3-b631-3c75cf50f04b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:14:59.889756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:14:59.890605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.891036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.891341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.892154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:14:59.894228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887618a-2cb2-487c-bbbd-6ac207097677 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:14:59.898600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:14:59.899306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.899596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.899877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.900684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:14:59.903291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758b464-715a-4b44-9c86-31733038eabb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:14:59.922235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 783/1547] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:14:59.923918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958037-6015-42f5-9cce-c93ea47c865d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.924477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958037-6015-42f5-9cce-c93ea47c865d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.924927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958037-6015-42f5-9cce-c93ea47c865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.934637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 765/1548] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:14:59.939233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525138d1-aea3-458a-8c1b-b5b232b7b88c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:14:59.939538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525138d1-aea3-458a-8c1b-b5b232b7b88c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:14:59.940244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525138d1-aea3-458a-8c1b-b5b232b7b88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:14:59.945911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2b958037-6015-42f5-9cce-c93ea47c865d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.947805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2b958037-6015-42f5-9cce-c93ea47c865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:14:59.963719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-525138d1-aea3-458a-8c1b-b5b232b7b88c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:14:59.964998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-525138d1-aea3-458a-8c1b-b5b232b7b88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.011897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 784/1549] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.017089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9a31-2f8b-4c99-a495-100a907dce0b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.017434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9a31-2f8b-4c99-a495-100a907dce0b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.017706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9a31-2f8b-4c99-a495-100a907dce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.024585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 766/1550] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.025885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.026180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.026462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.037849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-03fc9a31-2f8b-4c99-a495-100a907dce0b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.039139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-03fc9a31-2f8b-4c99-a495-100a907dce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.044271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.045450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.048243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 785/1551] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.049485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.049774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.050111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.058782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 767/1552] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.058782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.068383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0766643-b104-466f-9ab5-c616fbccfe7f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.072753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.073063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.073745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.073957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.074056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.074309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.075049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.076727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d431ff4-83a1-474f-84c5-008298501b58 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.084801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 786/1553] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.085967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.090234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 768/1554] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:00.090234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e060-80f9-457a-834e-61515faea0e1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.090234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e060-80f9-457a-834e-61515faea0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.090234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6e060-80f9-457a-834e-61515faea0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.100946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8fd46-3d4b-45d7-9fc2-91676ec0a317 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.106107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.106792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.107074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.107374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.108081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.109590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a4d6e060-80f9-457a-834e-61515faea0e1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.110875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b25e5-16e4-440c-bd80-1874482a0f22 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.111452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a4d6e060-80f9-457a-834e-61515faea0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.120573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 769/1555] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.122842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f44ce-eea4-4f50-aeba-e9fc8d6b7eec None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.123383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f44ce-eea4-4f50-aeba-e9fc8d6b7eec None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.124085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f44ce-eea4-4f50-aeba-e9fc8d6b7eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.138157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 787/1556] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/users => generated 290 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.139634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27cabd-cd5a-4ccc-9e26-1e6e032b1a72 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.140062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27cabd-cd5a-4ccc-9e26-1e6e032b1a72 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.140497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27cabd-cd5a-4ccc-9e26-1e6e032b1a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.145471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4f4f44ce-eea4-4f50-aeba-e9fc8d6b7eec None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.147265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4f4f44ce-eea4-4f50-aeba-e9fc8d6b7eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.158425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ac27cabd-cd5a-4ccc-9e26-1e6e032b1a72 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.159430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 770/1557] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.159605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ac27cabd-cd5a-4ccc-9e26-1e6e032b1a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.161050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.161824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.162131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.170824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 788/1558] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.172210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb805e0-7570-4b71-970b-de66a6e73208 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.172507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb805e0-7570-4b71-970b-de66a6e73208 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.172824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb805e0-7570-4b71-970b-de66a6e73208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.183398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.184873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.190299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8eb805e0-7570-4b71-970b-de66a6e73208 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.191653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8eb805e0-7570-4b71-970b-de66a6e73208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.192858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 771/1559] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.193738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.202204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 789/1560] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.203568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.203836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d2806-175b-4e62-87a9-2d0afc83d23e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.203958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.204165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.210435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.211123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.211409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.211528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.211687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.212415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.214969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1819e-6288-4ca9-8e16-4cbc0e3d023a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.223393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.224607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.226900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 772/1561] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:00.228193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2984cd-2519-4756-8442-7e827101682e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.228483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2984cd-2519-4756-8442-7e827101682e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.228785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2984cd-2519-4756-8442-7e827101682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.236413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 790/1562] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.237439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.237573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.248356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7f6ef-f2b9-4bb5-a711-b7e16a42f6ff None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.248847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0d2984cd-2519-4756-8442-7e827101682e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.250787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0d2984cd-2519-4756-8442-7e827101682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.253370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.254043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.254349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.254608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.255388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.257000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.258750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 773/1563] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.259538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2984cd-2519-4756-8442-7e827101682e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.269600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2984cd-2519-4756-8442-7e827101682e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.271410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 791/1564] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:00.272120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.274776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.275688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.275974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.276249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.276987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.277313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.278908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.282009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5843f-d9f1-40b6-8058-04de68698ab3 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.287739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.288389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.288669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.288935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.289641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.289948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.291226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 774/1565] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.291594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.292086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.303408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8709ae-e8d6-4756-a94a-3f485ae598a9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.304108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 792/1566] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.304923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.308375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.309217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.309643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.310056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.311347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.311928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.313545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.314413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365a006-afe0-4a19-bea1-17d8332c5f7f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.318934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.319612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.319894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.320169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.320873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.321181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.322791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.325600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 775/1567] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.326493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.341904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 793/1568] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.342102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996df8b-0d0c-4da6-a390-f8dc8d1512f0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.342188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.346632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.347565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.347849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.348121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] PATH_INFO: `/v3/projects/1b2cfea312a04240a0a0e709879d4832/users/ea7b896127b04912968d12a70b730884/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.354188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66858f21-07aa-4530-810c-fb5eed1699c9 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.359400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.360070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.360356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.360636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.361370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.361681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.363428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.364871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2cfea312a04240a0a0e709879d4832, user_id=ea7b896127b04912968d12a70b730884, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.367610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.376169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 794/1569] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.377265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.387405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb9edb-0584-4058-af87-f35cfa59982c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.389733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 776/1570] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:00 2022] PUT /identity/v3/projects/1b2cfea312a04240a0a0e709879d4832/users/ea7b896127b04912968d12a70b730884/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:00.390526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.452952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.453685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.453946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.454226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.455012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:00.455387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.457121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.457981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d316012-9bbb-445a-a97c-2a10be2ed2d3 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.463879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.464572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.464854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.465132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.465886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.467602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.469304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 795/1571] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:00.470193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.479408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 777/1572] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:00.480128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.481336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487c3f2-7945-46f4-b4fe-8e82d35f08e1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.486733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.487985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.488286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.488576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.489320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.491021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754c62a-c019-4b86-aebe-5f63bf61b0ec admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.492350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.495670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.496313 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.496602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.496892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.497571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.500039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.523905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 796/1573] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.523905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.530255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 778/1574] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.530255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.588836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe78d9e-0caf-4a20-9402-fd0f9ae14300 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.592861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b4f67-d04d-49f2-b39a-38b8ddb4e775 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.593599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.594272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.594562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.594811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.595543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.597643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.597808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.598484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.598757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.599033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.599753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.601647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b61084-2a43-4706-9711-8c1ecbb42828 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.624733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 797/1575] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.625865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.628011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 779/1576] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.629278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.629573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.629855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.632622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.642116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.657280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] MFA Rules not processed for user `ea7b896127b04912968d12a70b730884`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:00.658488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:00.690853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad36b7-a15f-4b6b-8862-520ea8948faa admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.724212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 780/1577] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:00.725049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.747645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.748427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.748596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.749037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.749721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.752258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.781474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 798/1578] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.785374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.793239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f34aa0-3715-455d-a2db-ac425104e89e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.846754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc5831-d53b-4bdc-b107-afb77418f13a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.850251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.850943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.851248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.851530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.852270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.853094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.854027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.854130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.854295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.854577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] PATH_INFO: `/v3/projects/f70ad89203de4b5cb4853fa4d0d6eeb4/users/3c8661e8aa094e839426aca1c5c5db62/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.869629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70ad89203de4b5cb4853fa4d0d6eeb4, user_id=3c8661e8aa094e839426aca1c5c5db62, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.870760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:15:00.870760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:15:00.870760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:15:00.872984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.885045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 781/1579] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.885045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.889881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 799/1580] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:00 2022] PUT /identity/v3/projects/f70ad89203de4b5cb4853fa4d0d6eeb4/users/3c8661e8aa094e839426aca1c5c5db62/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:00.900585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:00.914244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.919018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:00.947647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76af0b-90b9-4a9a-bfb7-c4ed62ed5520 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.959810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c79f5-df9c-4859-9419-cadece330e81 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:00.965137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:00.965827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:00.966528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:00.966810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:00.967603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:00.969567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:00.989976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 800/1581] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:00.990772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.000923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525b064-22ab-4ef8-8cde-658c46f52812 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.004441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.005139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.005335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.005467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.005700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.005988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.006272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.006525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.006619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.007247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.007593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:15:01.007593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:15:01.007593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:15:01.008807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.009115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c1771-063f-4434-9fdf-7b03a9eec6a8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.030280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 782/1582] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.030973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 801/1583] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:00 2022] POST /identity/v3/users => generated 284 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.034417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.034800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f774ca-39cb-425e-adf3-0132c72b2988 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.035118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f774ca-39cb-425e-adf3-0132c72b2988 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.035435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f774ca-39cb-425e-adf3-0132c72b2988 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.037059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 802/1584] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 30 04:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:01.037738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f774ca-39cb-425e-adf3-0132c72b2988 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.044675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b49c4-fb92-49cf-aa14-d14952ca1426 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.049346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.050018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.050307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.050586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.050927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f774ca-39cb-425e-adf3-0132c72b2988 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.051343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.053276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.056767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.057410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.057693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.057969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.058657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.060602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6742a-8fa1-41f7-ade8-6a73d5b1a27a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.074746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 783/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.075746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.083776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 803/1586] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.084944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3154c7c-2b65-4138-b1a0-da3d89c26321 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.085234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3154c7c-2b65-4138-b1a0-da3d89c26321 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.085356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55895965-915f-43e6-b169-330f2aa977ed admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.085521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3154c7c-2b65-4138-b1a0-da3d89c26321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.090109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.090985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.091322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.092440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.093205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.095259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.113814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a3154c7c-2b65-4138-b1a0-da3d89c26321 None None] MFA Rules not processed for user `3c8661e8aa094e839426aca1c5c5db62`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:01.115038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a3154c7c-2b65-4138-b1a0-da3d89c26321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:01.115611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 784/1587] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/users => generated 303 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.116503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.126405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f31cf5-709e-49f3-9002-d1e6c8347b3f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.131661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.132324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.132611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.132883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.133609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.135367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.146264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 785/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.147156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.157101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5af84f-0423-4a5a-b973-e5008f67a550 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.162466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.163146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.163439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.163720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.164440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.166014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.176477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 786/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.177329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.180636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 804/1590] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:01.181850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.182121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.182432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.187807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1a2c-3e53-432b-98b1-e6fab2cced1a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.192648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.193314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.193598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.193873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.194600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.196212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.200031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:01.201600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:01.206670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 787/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.210450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.219887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf2580-901c-43ae-afc2-ea949395cb7a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.224547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.225221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.225501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.225777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.226493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.228234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.238867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 788/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.239683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.249215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6df947-d145-4fdd-8ded-29338d5c5524 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.253858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.254497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.254778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.255135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.255808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.257385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.258586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:01.268763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 789/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.269762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.274130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 805/1594] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:01.274966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.285682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d61557-7482-498e-884e-ff769853f0b7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.285851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e359f-5398-4016-95a0-e96faf4131b0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.290578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.290741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.291273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.291396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.291554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.291668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.291832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.291934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.292560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.292678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.294190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.294296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.304509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 790/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.305011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 806/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.305799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.306638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.315333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb99d8-c8bf-4c12-8d21-deed76b90591 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.317533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98ed97-4fe2-4cdd-a49e-7d5901bbd375 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.320257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.320906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.321218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.321479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.322166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.322288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.322812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.323112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.323400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.324115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.324230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.325706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.336223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 791/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.337005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.346823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b9eae-642e-492d-93ee-79185cefc414 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.351409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.352053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.352334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.352609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.353318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.353424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 807/1598] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.355017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.357283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.365603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 792/1599] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.366586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.371566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:01.377453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047ec7-33a1-486e-b204-5d037974bacb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.419336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea2caa-05d9-42a7-8ef8-d3025a2ec133 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.423986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.424675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.424959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.425238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.425966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.427885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.436148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.436838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.437273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.437555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.438291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.438417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 808/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.439122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.439944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.453203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 793/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.453203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.462632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534c286-7cd8-492c-aab8-7617d7ac30dc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.467262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.468100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.468387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.468666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.469396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.470999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.480867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 794/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.481628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.491291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3d96d-c88d-427c-a317-f11e30f98324 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.495965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.496870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.497159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.497440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] PATH_INFO: `/v3/projects/d9abec1b410a4d089a04eb0024175b2e/users/03f46752fcf64283881170d63c76f424/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.512138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b2868-ed0b-4804-8b5a-3336be77d7c1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.512668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9abec1b410a4d089a04eb0024175b2e, user_id=03f46752fcf64283881170d63c76f424, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.515296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.518182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.518844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.519158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.519448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.529384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 795/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/d9abec1b410a4d089a04eb0024175b2e/users/03f46752fcf64283881170d63c76f424/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.530512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.584339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:01.586787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.588612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.593404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b0c8eb-fc0d-490f-b922-b64c30a7e00f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.599248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.599559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 809/1604] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:01.600241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.600579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.600723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.600865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] PATH_INFO: `/v3/projects/71588f8a5fc34919a77f28b6a8c8db0e/users/8e3bc7e015f5419aa956568707764f67/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.610543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf48e2-cb6b-4603-9edd-e7aa130d5a28 service nova] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.615386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.616119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.616472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.616630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71588f8a5fc34919a77f28b6a8c8db0e, user_id=8e3bc7e015f5419aa956568707764f67, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.616872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.617629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.619438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.619704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.636972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 796/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/71588f8a5fc34919a77f28b6a8c8db0e/users/8e3bc7e015f5419aa956568707764f67/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.637848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.644087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 810/1606] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:01.645098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.655110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735e374-0360-4bc2-961a-ec6bad2174db admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.692899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:01.709156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c33444-1165-4989-8826-2e7fcc2d11a0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.714580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.715520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.715816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.716100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] PATH_INFO: `/v3/projects/c9be5d4573e345fc9c1e4a065ed94c9a/users/19e1433fca7a49698d9b8eeac4648d15/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.720027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.720204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.720437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.720716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] PATH_INFO: `/v3/projects/6a37f11a6f794a518cfcfa600caf2445/users/392b8ea4f0c6480d9eb65ce93ec3e719/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.733670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9be5d4573e345fc9c1e4a065ed94c9a, user_id=19e1433fca7a49698d9b8eeac4648d15, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.736064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a37f11a6f794a518cfcfa600caf2445, user_id=392b8ea4f0c6480d9eb65ce93ec3e719, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.736686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.738758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.752366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 797/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/c9be5d4573e345fc9c1e4a065ed94c9a/users/19e1433fca7a49698d9b8eeac4648d15/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.756647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 811/1608] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/6a37f11a6f794a518cfcfa600caf2445/users/392b8ea4f0c6480d9eb65ce93ec3e719/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.756647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.756863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.827227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72ac01-2e8e-4148-b2b1-70ed82a12265 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.831211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd317-5633-42b8-95c8-75ea5a4581c4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.831940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.833048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.833338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.833624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] PATH_INFO: `/v3/projects/dc8c112787bf47f0bbf5d80c262cfd77/users/3c8a55c39d344398ba46ee8e9e2a2563/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.836401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.837312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.837601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.837878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] PATH_INFO: `/v3/projects/3e7ca13cb76941329dff62b26c65eb2f/users/b20e026e87ab45b293d26c8e53137f1b/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.848880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8c112787bf47f0bbf5d80c262cfd77, user_id=3c8a55c39d344398ba46ee8e9e2a2563, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.851562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df566f-acd3-40c7-8361-cb3362cd062e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.852690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e7ca13cb76941329dff62b26c65eb2f, user_id=b20e026e87ab45b293d26c8e53137f1b, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.855389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9212-2f24-4339-9bc6-aa7adcd159eb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.866521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 812/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/dc8c112787bf47f0bbf5d80c262cfd77/users/3c8a55c39d344398ba46ee8e9e2a2563/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.867756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bf99-9116-48d0-afab-82684d2187ca None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.868047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bf99-9116-48d0-afab-82684d2187ca None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.868326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720bf99-9116-48d0-afab-82684d2187ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.869289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 798/1610] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:01 2022] PUT /identity/v3/projects/3e7ca13cb76941329dff62b26c65eb2f/users/b20e026e87ab45b293d26c8e53137f1b/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:01.870538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.870927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.871303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.889324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:01.890557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:01.896295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1720bf99-9116-48d0-afab-82684d2187ca None None] MFA Rules not processed for user `03f46752fcf64283881170d63c76f424`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:01.897501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1720bf99-9116-48d0-afab-82684d2187ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:01.930358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:01.952750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 799/1611] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:01.952750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:01.960396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b91e8e-5f34-4320-8f74-400a1ac73bb3 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:01.965071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:01.965740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.966052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.966317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.967076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:01.971875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 813/1612] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:01.972051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665461d2-f1e5-4a52-a59e-906401687204 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:01.972165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb8ab-f7c9-4349-9422-7d95522a404e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.972165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb8ab-f7c9-4349-9422-7d95522a404e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.972165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bb8ab-f7c9-4349-9422-7d95522a404e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.985117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 800/1613] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:01.985117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:01.985117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:01.985117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:01.990923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2b6bb8ab-f7c9-4349-9422-7d95522a404e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:01.992191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2b6bb8ab-f7c9-4349-9422-7d95522a404e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:02.000643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 814/1614] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:02.001934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc22bd-380f-48e6-a8b0-6aeb1cfb2663 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.002225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc22bd-380f-48e6-a8b0-6aeb1cfb2663 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.002502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc22bd-380f-48e6-a8b0-6aeb1cfb2663 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.004071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 815/1615] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 30 04:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:02.005256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0f7b-0817-4a23-bc55-e1d6de5bf244 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.005507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0f7b-0817-4a23-bc55-e1d6de5bf244 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.005791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e0f7b-0817-4a23-bc55-e1d6de5bf244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.017224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] MFA Rules not processed for user `19e1433fca7a49698d9b8eeac4648d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:02.018608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:02.023308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c17e0f7b-0817-4a23-bc55-e1d6de5bf244 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:02.024472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c17e0f7b-0817-4a23-bc55-e1d6de5bf244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:02.031949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 816/1616] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:02.033162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.033455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.033735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.049980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:02.060136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] MFA Rules not processed for user `b20e026e87ab45b293d26c8e53137f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:02.061365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:02.084800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 801/1617] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:02.085568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.129588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 817/1618] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:02.130758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.141346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad414e7-d103-43ca-8366-6023542254f5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.143782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ec59b-ab5e-4ded-979b-8e4551b8a13e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.145962 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.146620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.146907 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.147206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.147402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.147897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.148034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.148332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.148616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.149827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1807e0-7b3c-46b2-84d0-6b0caafde1ed admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.157983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.159521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.170683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 802/1619] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:02.171743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.180209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 818/1620] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:02.180209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.180209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.180209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.182159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818fcda-6da7-43b9-bd27-1af21b417e90 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.200128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:02.201375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:02.245844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.246913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.247349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.247683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] PATH_INFO: `/v3/projects/38fc0a32fc9145898385dab18b623948/users/d78f642f27ad4162a496d56773d87219/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.266204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fc0a32fc9145898385dab18b623948, user_id=d78f642f27ad4162a496d56773d87219, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.269409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.279236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 819/1621] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:02.285312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.287236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 803/1622] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:02 2022] PUT /identity/v3/projects/38fc0a32fc9145898385dab18b623948/users/d78f642f27ad4162a496d56773d87219/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:02.288468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.307158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:02.349153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff62f6-a68c-4957-8e93-5c4d58b7ca8f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.358821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3d8dc-ef19-45dd-a1d4-996f75e79209 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.363540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.364242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.364674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.364856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.375190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:02.418017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.418806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.419470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.419912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.433950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.435499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.447693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 804/1623] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:02.448631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.516332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.518007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.530159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408893f6-838b-48c3-bfb3-5fcd9447b9ae service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.534575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.535385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.535756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.536677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.541893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 820/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:02.542872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.566430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bd711-ac07-4be6-9bea-1360efbac06f service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.574940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.575710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.576288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.576665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.577688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.591039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.630346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.632002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.644491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 805/1625] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:02.645553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.652116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 821/1626] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:02.653229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.659067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd7a7a-b7f4-44e6-a229-de23d3adba86 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.728875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.729712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.730064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.730442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.731176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.733750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.743433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:02.766107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05f667-697a-477b-bc87-071788a2c8e0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.771316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 806/1627] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:02.771316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.774982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.775864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.776239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.776745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.778229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.779419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.787589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:02.804318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 822/1628] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:02.805728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.828054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad82ec-e2a1-46b9-8a4e-43810590944b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.835903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.837278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.837953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.838400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.841305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.844881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.887004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 823/1629] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:02 2022] POST /identity/v3/users => generated 301 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:02.887920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.910983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabd12e-379f-4532-837f-59bf1df5db21 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.914673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78a3e3-a487-49ed-9e6c-59fec1846458 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:02.922187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.923897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.924325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.924613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.926602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:02.928149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:02.928484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:02.928726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:02.929540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:02.935603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:02.964887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 824/1630] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:02.966021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:02.990096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aefaf67-041a-496f-a563-4fe037be5233 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.001353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.003129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.003488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.003772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] PATH_INFO: `/v3/projects/38fc0a32fc9145898385dab18b623948/users/d78f642f27ad4162a496d56773d87219/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.009021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38fc0a32fc9145898385dab18b623948, user_id=d78f642f27ad4162a496d56773d87219, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.015982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.046655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 825/1631] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:02 2022] PUT /identity/v3/projects/38fc0a32fc9145898385dab18b623948/users/d78f642f27ad4162a496d56773d87219/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:03.047850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.095263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.097812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.123461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 807/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:03.131763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.167652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3f222-1b43-4bf5-a8ec-6b28e4485b72 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.218852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ae8d2-be97-4f3c-9efe-9d1c7d23068c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.224586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.225584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.226070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.226496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.227454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.229261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2a9b2-18df-4dcc-a50f-91d23e3df6f2 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.243212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 826/1633] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:03.245214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.245913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.246650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.277171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] MFA Rules not processed for user `d78f642f27ad4162a496d56773d87219`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:03.278706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:03.318798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.320790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.321080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.321360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] PATH_INFO: `/v3/projects/116e0db57f154ae1ba738e7f65529fff/users/8d4f86fda9a24e7582f6ad1946eba93a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.358793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=116e0db57f154ae1ba738e7f65529fff, user_id=8d4f86fda9a24e7582f6ad1946eba93a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.363886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.390275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 808/1634] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:03 2022] PUT /identity/v3/projects/116e0db57f154ae1ba738e7f65529fff/users/8d4f86fda9a24e7582f6ad1946eba93a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:03.395124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.435704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 827/1635] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:03.436694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.440698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:03.466014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e003f-d04f-4cdc-9dde-5088e4d78f48 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.525693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:03.557700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.559011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.559325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.561960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.562944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.563986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc85f51-2f75-4680-a9c9-c9d18dcc6cde admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.564615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.574704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.575846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.576100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.576449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] PATH_INFO: `/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6/users/1d1e5b3fd9844950bccf266c01a4efa1/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.589170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 828/1636] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:03.596790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.597462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0237c9ae764f1a9ba8ec781c475ff6, user_id=1d1e5b3fd9844950bccf266c01a4efa1, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.600294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.616167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8a38-de6b-41d7-9ced-cc5c98a046cc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.628888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 809/1637] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:03 2022] PUT /identity/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6/users/1d1e5b3fd9844950bccf266c01a4efa1/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:03.631223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.744126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.745139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.745486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.745826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] PATH_INFO: `/v3/projects/116e0db57f154ae1ba738e7f65529fff/users/8d4f86fda9a24e7582f6ad1946eba93a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.752074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116e0db57f154ae1ba738e7f65529fff, user_id=8d4f86fda9a24e7582f6ad1946eba93a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.755121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.791476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd47bda-e8a3-4d5e-8c3f-56b0742ed9df admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.793054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 829/1638] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:03 2022] PUT /identity/v3/projects/116e0db57f154ae1ba738e7f65529fff/users/8d4f86fda9a24e7582f6ad1946eba93a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:03.795550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:03.920108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276176c-9c89-46a3-a450-12a2583399d3 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:03.929048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.929848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.930230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.930599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.938646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:03.939502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.939644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.942237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:03.943028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:03.945667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8afe6-c76a-4e03-99f8-64cd9f1b2256 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:03.976618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 810/1639] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:03.998487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:03.998487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:03.998487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.060185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.063126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.063294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] MFA Rules not processed for user `8d4f86fda9a24e7582f6ad1946eba93a`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:04.064341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:04.088220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 830/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:04.089903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.117233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74634d19-17ee-43b5-97d3-0ac2ec1e59e1 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:04.123175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:04.136794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.138370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.140184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] PATH_INFO: `/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6/users/1d1e5b3fd9844950bccf266c01a4efa1/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.145815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0237c9ae764f1a9ba8ec781c475ff6, user_id=1d1e5b3fd9844950bccf266c01a4efa1, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.155503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336a6a5-0096-4ec6-94fd-f05fce94c83c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.177689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:04.198259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 831/1641] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:04 2022] PUT /identity/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6/users/1d1e5b3fd9844950bccf266c01a4efa1/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:04.200004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda0d48-a03b-4b64-a15b-547249c32668 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.201664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda0d48-a03b-4b64-a15b-547249c32668 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.204129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda0d48-a03b-4b64-a15b-547249c32668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.238376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:04.299264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5bda0d48-a03b-4b64-a15b-547249c32668 None None] MFA Rules not processed for user `1d1e5b3fd9844950bccf266c01a4efa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:04.304815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5bda0d48-a03b-4b64-a15b-547249c32668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:04.316188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 811/1642] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:04.333381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.356918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb46fa6-7138-4c37-9be5-8ab9f4018d6f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:04.487639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 832/1643] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:04.493767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.494093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.494333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.509038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:04.511804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.512797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.513604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.533634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] MFA Rules not processed for user `3c8a55c39d344398ba46ee8e9e2a2563`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:04.535023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:04.635294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 833/1644] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:04 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:04.636400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.650463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ea63-cbc0-40a6-b7de-dbafdec77ed1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:04.654639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:04.656215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.656757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.657038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.689973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.691576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.704287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 812/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:04.705111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.774219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.776095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd18be9-6888-4b60-917c-780f676f9baa service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.787636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 834/1646] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:04.789330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e95a7-12f2-4237-a25a-1ac759e51736 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.790120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e95a7-12f2-4237-a25a-1ac759e51736 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.790553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e95a7-12f2-4237-a25a-1ac759e51736 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.792975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 835/1647] 10.222.0.13 () {54 vars in 961 bytes} [Thu Jun 30 04:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:04.794001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e95a7-12f2-4237-a25a-1ac759e51736 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.811661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e95a7-12f2-4237-a25a-1ac759e51736 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:04.830849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:04.833439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.834360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.835198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.838776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c769882-c504-40d8-bbe8-e225abed01c0 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:04.851537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:04.852238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.852522 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.852800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.853705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.855579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.863492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:04.865517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83010db8-a75f-403e-8641-cbbb1c970c9f service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:04.909146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 836/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:04.910763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:04.911010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:04.911682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:04.921315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 813/1649] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:04 2022] POST /identity/v3/projects => generated 403 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:04.922940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:04.937781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:04.960431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:04.988795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:04.990084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:05.122360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd25d5f-ebd8-480f-9cd0-ce399241199e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.129240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.130021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.130310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.130630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.131506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.134671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.167654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 814/1650] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:04 2022] POST /identity/v3/users => generated 308 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:05.168635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.234790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 837/1651] 10.222.0.13 () {58 vars in 1058 bytes} [Thu Jun 30 04:15:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:05.237319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.266212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7251083-af9a-4b23-a55c-b2cdaadd3252 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.288010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.288844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.290054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.290054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.290415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.292371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99dd277-432e-4d73-8a36-d4cd09d98556 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.300524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16038f90-163d-458b-b0d1-e2f62a867ec0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.311445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.312284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.312574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.312858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.314737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 838/1652] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:05.317597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1483ba4-ed79-4d64-ab66-f6ce47e48858 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.319525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1483ba4-ed79-4d64-ab66-f6ce47e48858 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.319816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1483ba4-ed79-4d64-ab66-f6ce47e48858 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.324010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 839/1653] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 30 04:15:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:05.324833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1483ba4-ed79-4d64-ab66-f6ce47e48858 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.352389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1483ba4-ed79-4d64-ab66-f6ce47e48858 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.363083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.365051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.365451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.365828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] PATH_INFO: `/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f/users/4d7e0f68c67a4c37a3405025ccbfb023/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.399556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20f679fb7c749f4bdbcc34c68b9256f, user_id=4d7e0f68c67a4c37a3405025ccbfb023, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.403337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.418399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.420059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a050c-2cab-4bf0-8979-5573ca24e4ff service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.427997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 840/1654] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:05 2022] PUT /identity/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f/users/4d7e0f68c67a4c37a3405025ccbfb023/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:05.427997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.431583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 815/1655] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:05.433124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.433520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.433920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.454319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:05.455795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:05.534832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1062b3-d3e3-40e3-8e1f-87ca85d10915 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.545692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.546897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.548067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.548196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:05.548535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.609017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 816/1656] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:15:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:05.612149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.694558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.696060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.736751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 841/1657] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:05.787424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471d6c7-15f1-4095-b2ba-c96128ab7aad None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.802942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.803952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.804394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.804828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.805756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.810428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.839068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 817/1658] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:05.843343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:05.856349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6879f5bf-912c-4678-b93c-eb2667f62a37 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:05.870695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:05.871797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:05.872654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:05.872654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] PATH_INFO: `/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f/users/4d7e0f68c67a4c37a3405025ccbfb023/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:05.879945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20f679fb7c749f4bdbcc34c68b9256f, user_id=4d7e0f68c67a4c37a3405025ccbfb023, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:05.883222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ea224-0ad5-441a-b82c-4f702da276fe admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:05.928104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 842/1659] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:05 2022] PUT /identity/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f/users/4d7e0f68c67a4c37a3405025ccbfb023/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:06.046435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:06.062082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f541322-27d2-4776-b632-2e27ffa6e4e5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:06.166515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:06.167971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:06.169270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:06.169955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:06.360835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:06.365953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590912-183f-4536-a232-57b74777b2ac service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:06.400337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 818/1660] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:08.362060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e51516-b2bc-4040-99c9-20ebddc7e5ea None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:08.362641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e51516-b2bc-4040-99c9-20ebddc7e5ea None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:08.362839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e51516-b2bc-4040-99c9-20ebddc7e5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:08.371023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:08.448711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-17e51516-b2bc-4040-99c9-20ebddc7e5ea None None] MFA Rules not processed for user `4d7e0f68c67a4c37a3405025ccbfb023`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:08.452978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-17e51516-b2bc-4040-99c9-20ebddc7e5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:08.697022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 843/1661] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:10.304900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcda66c-9580-4cc9-8363-b7c603aa8c41 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:10.305478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcda66c-9580-4cc9-8363-b7c603aa8c41 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:10.305478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcda66c-9580-4cc9-8363-b7c603aa8c41 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:10.307755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 819/1662] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:10.323683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dda12-c2f8-4d7d-80bd-e5bf301a0108 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:10.324753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dda12-c2f8-4d7d-80bd-e5bf301a0108 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:10.325163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dda12-c2f8-4d7d-80bd-e5bf301a0108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:10.359831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a16dda12-c2f8-4d7d-80bd-e5bf301a0108 None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:10.361489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a16dda12-c2f8-4d7d-80bd-e5bf301a0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:10.507521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 844/1663] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:15:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:10.526473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315470-546c-4a1b-8ae0-9e809aa8b3e9 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:10.527036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315470-546c-4a1b-8ae0-9e809aa8b3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:10.527621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315470-546c-4a1b-8ae0-9e809aa8b3e9 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:10.529442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 820/1664] 10.222.0.13 () {54 vars in 960 bytes} [Thu Jun 30 04:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:10.539491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:10.541702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:10.542440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:10.573597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:10.576202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:10.592572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 845/1665] 10.222.0.13 () {58 vars in 1057 bytes} [Thu Jun 30 04:15:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:10.611035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315470-546c-4a1b-8ae0-9e809aa8b3e9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:10.653817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315470-546c-4a1b-8ae0-9e809aa8b3e9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:10.660787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:10.662234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:10.662514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:10.662902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:10.704278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:10.710595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:10.744315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 821/1666] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:11.460546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:11.572435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709686ca-7d9b-4c0f-b32c-491b98c79f87 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:11.590995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:11.592521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:11.592746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:11.593064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] PATH_INFO: `/v3/users/3c8a55c39d344398ba46ee8e9e2a2563` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:11.594367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8a55c39d344398ba46ee8e9e2a2563)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:11.595515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: WARNING py.warnings [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:15:11.595515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: warnings.warn(msg) Jun 30 04:15:11.595515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: Jun 30 04:15:11.597263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:11.637742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:11.697129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8a55c39d344398ba46ee8e9e2a2563'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:11.749200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8a55c39d344398ba46ee8e9e2a2563'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:11.766688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8a55c39d344398ba46ee8e9e2a2563'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:11.767630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8a55c39d344398ba46ee8e9e2a2563'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:11.769103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8a55c39d344398ba46ee8e9e2a2563'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:11.795539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 846/1667] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:11 2022] DELETE /identity/v3/users/3c8a55c39d344398ba46ee8e9e2a2563 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:12.065483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:12.072584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:12.089634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2f59-a894-4c6f-9378-5363f1075499 service placement] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:12.266933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:12.268663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:12.269144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:12.269824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] PATH_INFO: `/v3/projects/dc8c112787bf47f0bbf5d80c262cfd77` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:12.274599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8c112787bf47f0bbf5d80c262cfd77)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:12.277654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: WARNING py.warnings [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 04:15:12.277654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: warnings.warn(msg) Jun 30 04:15:12.277654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: Jun 30 04:15:12.281392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:12.430705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8c112787bf47f0bbf5d80c262cfd77', 'request_id': 'req-cba3e930-9d5e-4f9f-b455-468ae7daad8d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.453009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8c112787bf47f0bbf5d80c262cfd77', 'request_id': 'req-cba3e930-9d5e-4f9f-b455-468ae7daad8d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.473225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8c112787bf47f0bbf5d80c262cfd77', 'request_id': 'req-cba3e930-9d5e-4f9f-b455-468ae7daad8d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.517262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] The token cache is being invalidate because project dc8c112787bf47f0bbf5d80c262cfd77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:12.517520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.518202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cba3e930-9d5e-4f9f-b455-468ae7daad8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.520427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 822/1668] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:12 2022] DELETE /identity/v3/projects/dc8c112787bf47f0bbf5d80c262cfd77 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:12.524492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:12.527150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:12.556746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2110552-d7e6-48f5-98c1-55be698d4745 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:12.664141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:12.665194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:12.665575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:12.665969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] PATH_INFO: `/v3/users/4d7e0f68c67a4c37a3405025ccbfb023` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:12.667782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7e0f68c67a4c37a3405025ccbfb023)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:12.674074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:12.742325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7e0f68c67a4c37a3405025ccbfb023'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.802041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7e0f68c67a4c37a3405025ccbfb023'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.804385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:12.804681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:12.804964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:12.820617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7e0f68c67a4c37a3405025ccbfb023'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.821247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7e0f68c67a4c37a3405025ccbfb023'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.821875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1823e514-0fa5-49f4-b214-169ddbb7ed3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7e0f68c67a4c37a3405025ccbfb023'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:12.854134 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 847/1669] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:12 2022] DELETE /identity/v3/users/4d7e0f68c67a4c37a3405025ccbfb023 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:12.864910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] MFA Rules not processed for user `8e3bc7e015f5419aa956568707764f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:12.866153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:12.868538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53edd-db63-46cd-b7f8-f26e16866471 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:12.869487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53edd-db63-46cd-b7f8-f26e16866471 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:12.869487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53edd-db63-46cd-b7f8-f26e16866471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:12.924539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c3c53edd-db63-46cd-b7f8-f26e16866471 None None] MFA Rules not processed for user `392b8ea4f0c6480d9eb65ce93ec3e719`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:12.925921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c3c53edd-db63-46cd-b7f8-f26e16866471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:12.936839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:12.984965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 823/1670] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:13.000162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.001906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.067210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:13.087155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 848/1671] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:13.098373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.098800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.099300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.125167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] MFA Rules not processed for user `392b8ea4f0c6480d9eb65ce93ec3e719`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:13.127153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.142970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 849/1672] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:13.159869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.166588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.198047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d64d0a-1395-43b3-87ab-dc88a527d1a3 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:13.207389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:13.208954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.209454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.210311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.240599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:13.243147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:13.263365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 824/1673] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:13.264304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.266614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.308500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274addf-a484-4bae-8e69-137dfd825f21 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:13.314916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35fddb-97a6-4533-8fd4-036a493b86f6 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:13.320519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:13.321551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.327250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.328763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] PATH_INFO: `/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.331921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20f679fb7c749f4bdbcc34c68b9256f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:13.332483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:13.333228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.333520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.333799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.334220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:13.353584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:13.355251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abc4a1-b10a-4d4d-9c3b-747d764ee26c service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:13.377110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 825/1674] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:13.514022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20f679fb7c749f4bdbcc34c68b9256f', 'request_id': 'req-6d3c9fff-3816-410a-bd7f-67c55d87c277'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:13.541862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20f679fb7c749f4bdbcc34c68b9256f', 'request_id': 'req-6d3c9fff-3816-410a-bd7f-67c55d87c277'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:13.555942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20f679fb7c749f4bdbcc34c68b9256f', 'request_id': 'req-6d3c9fff-3816-410a-bd7f-67c55d87c277'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:13.628385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] The token cache is being invalidate because project f20f679fb7c749f4bdbcc34c68b9256f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:13.628854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:13.629679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6d3c9fff-3816-410a-bd7f-67c55d87c277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:13.637759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 850/1675] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:13 2022] DELETE /identity/v3/projects/f20f679fb7c749f4bdbcc34c68b9256f => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:13.639575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.640210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.640526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.689035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:13.689035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.784779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 826/1676] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:13.788438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.788925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.789398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.817705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:13.819751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:13.831801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 851/1677] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:13.836441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.851130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4fc5b-278f-400a-8d8d-4cb8e029365b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:13.864495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:13.865909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.865909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.865909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.866489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:13.866805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:13.868637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:13.896700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 827/1678] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:13.901430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.917759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7d8c7-49de-47f2-ba61-415bf71b781c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:13.924090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:13.924444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:13.924731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:13.925009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:13.925739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:13.930770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:13.974261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 852/1679] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:13 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:13.976315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:13.988111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53af73c-0b13-4879-8517-1014d649de05 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:14.094493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:14.096315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.096783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.097234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.098130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:14.100986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:14.135787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 828/1680] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:13 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:14.140722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:14.148233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:14.157929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe69ff4-dd2d-4a91-8ad6-4d5e38619afc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:14.178987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:14.180157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.180684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.182834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.183992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:14.186105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:14.220811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 853/1681] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:14.220995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:14.242305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab077c4-56f0-406d-b6c3-a4590bc78b0d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:14.254612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:14.255571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.255865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.256144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.256903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:14.265269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94768ae4-da5e-4e3a-a2b7-bf100d5efe35 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:14.285718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:14.309464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:14.317587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 829/1682] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:14.322772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d400a-f67e-4be9-9ef7-e377e6e2292f None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.327548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981ed52-76db-4704-80e6-33a8ba772424 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:14.328019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d400a-f67e-4be9-9ef7-e377e6e2292f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.329117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d400a-f67e-4be9-9ef7-e377e6e2292f None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.331675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 830/1683] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 30 04:15:14 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:14.338538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.339040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.339372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.342878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:14.354847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.355502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.355795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] PATH_INFO: `/v3/projects/14c8ea32df0041e89f6a8dad4007d7b8/users/cfaf12d12d6a423bb97dd5353e98bf67/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.398366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c8ea32df0041e89f6a8dad4007d7b8, user_id=cfaf12d12d6a423bb97dd5353e98bf67, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:14.401316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1a5ad-2514-4328-bd25-364e8d390f91 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:14.429680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:14.436864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:14.456898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 854/1684] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:14 2022] PUT /identity/v3/projects/14c8ea32df0041e89f6a8dad4007d7b8/users/cfaf12d12d6a423bb97dd5353e98bf67/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:14.471435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f391ab-5058-438e-8253-35c147782942 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.471908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f391ab-5058-438e-8253-35c147782942 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.472212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f391ab-5058-438e-8253-35c147782942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.552076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d3f391ab-5058-438e-8253-35c147782942 None None] MFA Rules not processed for user `cfaf12d12d6a423bb97dd5353e98bf67`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:14.554763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d3f391ab-5058-438e-8253-35c147782942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:14.623297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 831/1685] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:15:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:14.632851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:14.634496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:14.649341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed5641-6d5f-4e8d-a0a2-253faf911c42 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:14.654764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:14.655707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:14.655872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:14.657637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:14.672784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:14.674495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:14.689319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 832/1686] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:14.760258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:14.761190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 855/1687] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:14 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:15.018142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f39d98-753a-4525-b472-e23ee5169f8e service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:15.041877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:15.042580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:15.042867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:15.043174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:15.043880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:15.047170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:15.128178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 833/1688] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:14 2022] POST /identity/v3/projects => generated 369 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:15.137528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f391ab-5058-438e-8253-35c147782942 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:15.206943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f391ab-5058-438e-8253-35c147782942 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:15.463663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:15.475332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:15.476064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:15.480696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:15.501204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:15.521576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:15.652401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:15.657381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 856/1689] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:15 2022] POST /identity/v3/users => generated 291 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:15.667516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85231e-81e1-44a7-a36c-d372c3c3abdb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:15.707904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:15.708607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:15.708894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:15.709174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:15.709916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:15.711580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:15.722509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:15.724454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:15.746456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 834/1690] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:15.751157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:15.793380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ed702-a61d-490f-b9de-ac30f64e0ac6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:15.825032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:15.826533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:15.827871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:15.828365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:15.828626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] PATH_INFO: `/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6/users/94040d6963e946459856a3217c98e3d8/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:15.881356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d676670-1227-4de8-99f7-42e55b742c08 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:15.882646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c813fd6842e14a4cb2ee595c9512c7d6, user_id=94040d6963e946459856a3217c98e3d8, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:15.888417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:15.895822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:15.897137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:15.897581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:15.898044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:15.904890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:15.923582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 835/1691] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:15 2022] PUT /identity/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6/users/94040d6963e946459856a3217c98e3d8/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:15.928699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:15.974165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:15.975823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:16.013919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 857/1692] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:16.045706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:16.123477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466b638-adb0-4cdd-b614-42affedf5520 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:16.145158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:16.145733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.145973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.146320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.147325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:16.153321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201e619-2097-4510-b912-3e256930c282 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:16.180394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 836/1693] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:16.185064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:16.199591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f44ec-a02e-4535-8416-5c765cfc4065 service placement] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:16.210331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:16.213263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.213561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.214976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] PATH_INFO: `/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6/users/94040d6963e946459856a3217c98e3d8/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.219419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c813fd6842e14a4cb2ee595c9512c7d6, user_id=94040d6963e946459856a3217c98e3d8, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:16.225676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d608aa-6df5-4d9d-98ff-02730698605a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:16.264895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 858/1694] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:16 2022] PUT /identity/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6/users/94040d6963e946459856a3217c98e3d8/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:16.275590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.276102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.276555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.346967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] MFA Rules not processed for user `94040d6963e946459856a3217c98e3d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:16.350478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:16.482861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 837/1695] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:16.507645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.508233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.508738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.529349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:16.531411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:16.578068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] MFA Rules not processed for user `8e3bc7e015f5419aa956568707764f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:16.589285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:16.689212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebf93c-2bad-4384-afa0-d7f65c66643e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:16.695511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:16.696461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.696752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.697034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] PATH_INFO: `/v3/users/b20e026e87ab45b293d26c8e53137f1b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.699650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20e026e87ab45b293d26c8e53137f1b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:16.707189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:16.710864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 859/1696] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:16.711922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:16.724322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:16.802856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20e026e87ab45b293d26c8e53137f1b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:16.813801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20e026e87ab45b293d26c8e53137f1b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:16.815807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:16.862329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20e026e87ab45b293d26c8e53137f1b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:16.864111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20e026e87ab45b293d26c8e53137f1b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:16.870853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fcbff-d010-4c7a-b8b9-beba277c76c3 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:16.882053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-613d4c97-8607-4edc-884f-2e3e932fc2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20e026e87ab45b293d26c8e53137f1b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:16.882258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:16.882805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.883147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.883444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.885763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:16.914588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:16.916116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:16.922248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 838/1697] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:16 2022] DELETE /identity/v3/users/b20e026e87ab45b293d26c8e53137f1b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:16.924233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:16.924596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:16.924910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:16.957338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 860/1698] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:16.993664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:16.997886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:17.133594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:17.140123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:17.168518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1be5-ba2c-47a9-aa0b-23406b790812 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:17.183219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 839/1699] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:17.193678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:17.195868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:17.331673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:17.332502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:17.332785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:17.333064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:17.335182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:17.365353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed777d-67ba-4745-8141-67988d0169b5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:17.373753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:17.374434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:17.375458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:17.375458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:17.393273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:17.397408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:17.398949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:17.425600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 840/1700] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:17.474346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:17.475847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:17.498773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 861/1701] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:17 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:18.006097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:18.007710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:18.027529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be0933-b8c0-4ec6-b53e-b79ce91c6e26 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:18.037454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:18.038303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:18.038588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:18.038868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] PATH_INFO: `/v3/projects/3e7ca13cb76941329dff62b26c65eb2f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:18.043859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e7ca13cb76941329dff62b26c65eb2f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:18.047088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:18.199302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e7ca13cb76941329dff62b26c65eb2f', 'request_id': 'req-73888526-45d5-4bf3-b3a4-c640285d980d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.213154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e7ca13cb76941329dff62b26c65eb2f', 'request_id': 'req-73888526-45d5-4bf3-b3a4-c640285d980d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.221508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e7ca13cb76941329dff62b26c65eb2f', 'request_id': 'req-73888526-45d5-4bf3-b3a4-c640285d980d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.232201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:18.239399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] The token cache is being invalidate because project 3e7ca13cb76941329dff62b26c65eb2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:18.240496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.243085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.245447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 841/1702] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:18 2022] DELETE /identity/v3/projects/3e7ca13cb76941329dff62b26c65eb2f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:18.249454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:18.262108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdda6c-d909-494f-85d3-625d93846fbe service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:18.369420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:18.371331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:18.371637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:18.372059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] PATH_INFO: `/v3/users/1d1e5b3fd9844950bccf266c01a4efa1` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:18.375077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d1e5b3fd9844950bccf266c01a4efa1)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:18.377890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:18.509071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d1e5b3fd9844950bccf266c01a4efa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.533213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d1e5b3fd9844950bccf266c01a4efa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.572141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d1e5b3fd9844950bccf266c01a4efa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.572921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d1e5b3fd9844950bccf266c01a4efa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.573590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a18f412d-d7d3-42dd-8176-21ebceb01e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d1e5b3fd9844950bccf266c01a4efa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:18.594477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 862/1703] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:18 2022] DELETE /identity/v3/users/1d1e5b3fd9844950bccf266c01a4efa1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:19.102741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:19.104867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:19.137854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888526-45d5-4bf3-b3a4-c640285d980d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:19.232520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:19.235779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:19.235779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:19.235779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] PATH_INFO: `/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:19.247367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0237c9ae764f1a9ba8ec781c475ff6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:19.250098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:19.377082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0237c9ae764f1a9ba8ec781c475ff6', 'request_id': 'req-2c4911df-69e4-448a-96c8-a02c1ac87bf5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:19.388622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0237c9ae764f1a9ba8ec781c475ff6', 'request_id': 'req-2c4911df-69e4-448a-96c8-a02c1ac87bf5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:19.398619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0237c9ae764f1a9ba8ec781c475ff6', 'request_id': 'req-2c4911df-69e4-448a-96c8-a02c1ac87bf5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:19.445615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] The token cache is being invalidate because project 3a0237c9ae764f1a9ba8ec781c475ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:19.446829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:19.449723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c4911df-69e4-448a-96c8-a02c1ac87bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:19.455690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 842/1704] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:19 2022] DELETE /identity/v3/projects/3a0237c9ae764f1a9ba8ec781c475ff6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:19.461737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:19.462261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:19.462416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:19.499120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:19.500859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:19.633015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:19.685801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 863/1705] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:19.693838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:19.694153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:19.694411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:19.719125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:19.720800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:19.736268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 843/1706] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:19.736377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:19.760261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717e29c-133b-472a-9d30-cb28c7324967 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:19.767927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:19.768703 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:19.768995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:19.769303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:19.770055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:19.770365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:19.772691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:19.795627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 864/1707] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:15:19.799616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:19.833010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8fac2-b109-478e-b9c8-c5f78edb3e2c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:19.847482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:19.847883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:19.848172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:19.848448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:19.849194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:19.851315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:19.915728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 844/1708] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:19 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:19.920879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:19.936485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499b7dc-9644-4c41-a00f-ea80c6fc57d8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.025822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.026802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.028797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.029105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.030013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.032561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.066611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 865/1709] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:19 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:20.067908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.085310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b406c-6598-4b46-8cae-e03a6d4a55bf admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.097511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.098504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.098891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.099479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.100301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.102456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.129804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 845/1710] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:20.134577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.147188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7f07d-ae85-450d-bad4-bcf983b5e630 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.162012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.162643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.162643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.162942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.164526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.165646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461961f-faf1-4f4c-a6c0-48c3a9d84971 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.195588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 866/1711] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:20.197217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.220490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcf063-79cf-41ca-8c16-7554d7e05638 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.242694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.243812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.244157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.244735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] PATH_INFO: `/v3/projects/ddd0bdde782b44a8957ab7b8ab2d38bf/users/fea8685db2144c208afe1c9de7c9af80/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.279014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd0bdde782b44a8957ab7b8ab2d38bf, user_id=fea8685db2144c208afe1c9de7c9af80, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.287251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.322366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 846/1712] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:20 2022] PUT /identity/v3/projects/ddd0bdde782b44a8957ab7b8ab2d38bf/users/fea8685db2144c208afe1c9de7c9af80/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:20.323621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.323917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.324198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.378923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] MFA Rules not processed for user `fea8685db2144c208afe1c9de7c9af80`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:20.380455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:20.588325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 867/1713] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:20.592849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.643803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:20.698078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeef19f-58cb-4926-8a22-9794182b3f4e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.705275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.705275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.705275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.705704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.706479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.708932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.741303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 847/1714] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:20 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:20.745588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.763503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:20.776564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4ed9a-29e4-4bd8-9bfb-a02a611fd0c8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.778006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:20.903497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:20.950745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5615332-3994-47f5-952e-58505f9d5d1b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:20.963485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.964489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.964768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.965122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] PATH_INFO: `/v3/users/ea7b896127b04912968d12a70b730884` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.968132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7b896127b04912968d12a70b730884)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.968306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:20.968306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:20.968306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:20.968533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:20.968810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:20.969240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:20.971472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:21.066592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 868/1715] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:20 2022] POST /identity/v3/users => generated 301 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:21.068936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.110669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef4f1b-1474-4192-950c-fff3f4c9a383 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.134181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.134887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.135208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.135527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.136249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:21.141131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf501c2-4cc3-4553-989a-91e281d973e3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:21.141527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7b896127b04912968d12a70b730884'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:21.163026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7b896127b04912968d12a70b730884'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:21.182693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 869/1716] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:21.184093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad72f-adcb-4461-9977-46287c296590 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.184386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad72f-adcb-4461-9977-46287c296590 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.184668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ad72f-adcb-4461-9977-46287c296590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.222903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7b896127b04912968d12a70b730884'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:21.223985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-771ad72f-adcb-4461-9977-46287c296590 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:21.225041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7b896127b04912968d12a70b730884'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:21.225226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-771ad72f-adcb-4461-9977-46287c296590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:21.251078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7b896127b04912968d12a70b730884'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:21.259927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 848/1717] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:20 2022] DELETE /identity/v3/users/ea7b896127b04912968d12a70b730884 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:21.264532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.272136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:21.348051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:21.434086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 870/1718] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:15:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:21.439655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.442066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7aa76-1f45-44d4-8f84-2659accb1329 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.442333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.445001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.454551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.456182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.458778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.459200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] PATH_INFO: `/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98/users/5b6e7e9c34084a569a1cf0609c2800e6/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.497696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9734d3f95b9f48aaa7f96c47fad4cd98, user_id=5b6e7e9c34084a569a1cf0609c2800e6, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:21.503084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:21.520201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:21.528794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:21.545754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 849/1719] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:21 2022] PUT /identity/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98/users/5b6e7e9c34084a569a1cf0609c2800e6/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:21.547452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.550183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 871/1720] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:21.625291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.626810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:21.654809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fb935-1357-4062-a9fe-b52b17388bb5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.667154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.667875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.668164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.668446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.669184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:21.680293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:21.703295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 850/1721] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:21.719432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.733030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e49642-096c-4fda-8236-2d3f88c77523 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.741005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.742222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.742685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.743207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] PATH_INFO: `/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98/users/5b6e7e9c34084a569a1cf0609c2800e6/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.745855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9734d3f95b9f48aaa7f96c47fad4cd98, user_id=5b6e7e9c34084a569a1cf0609c2800e6, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:21.750619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:21.767189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 851/1722] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:21 2022] PUT /identity/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98/users/5b6e7e9c34084a569a1cf0609c2800e6/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:21.768575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:21.770650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:21.821218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7baa3-c773-4bcb-bc75-9bc26f3f2374 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.882164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d79b46-a0cb-4e55-9e54-f43ae40b34ed admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:21.891421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.892423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.892695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.892979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.945010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:21.946152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:21.946493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:21.946825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:21.990740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:21.993928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:22.037085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:22.038556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748ef22-06ac-428f-b998-b788dff8b402 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:22.055695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 852/1723] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:15:22.060120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6afc-9c9f-422a-97f4-493d0eb25945 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:22.061056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6afc-9c9f-422a-97f4-493d0eb25945 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:22.061056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf6afc-9c9f-422a-97f4-493d0eb25945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:22.064006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:22.065611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:22.082007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 872/1724] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:22.082951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:22.084726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.104242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116e90-ae69-46b8-9709-a987422bc2ec service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:22.110968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:22.114084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:22.114567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:22.115122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] PATH_INFO: `/v3/users/19e1433fca7a49698d9b8eeac4648d15` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:22.117666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e1433fca7a49698d9b8eeac4648d15)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:22.121432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:22.136271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-60bf6afc-9c9f-422a-97f4-493d0eb25945 None None] MFA Rules not processed for user `5b6e7e9c34084a569a1cf0609c2800e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:22.137713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-60bf6afc-9c9f-422a-97f4-493d0eb25945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.278377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e1433fca7a49698d9b8eeac4648d15'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:22.306624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 853/1725] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:22.330865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e1433fca7a49698d9b8eeac4648d15'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:22.355642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e1433fca7a49698d9b8eeac4648d15'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:22.357141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e1433fca7a49698d9b8eeac4648d15'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:22.362383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e1433fca7a49698d9b8eeac4648d15'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:22.408199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 873/1726] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:22 2022] DELETE /identity/v3/users/19e1433fca7a49698d9b8eeac4648d15 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:22.413615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:22.416454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:22.416454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:22.459300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:22.460984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.470988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:22.472575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.593671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3505080-beeb-4bdb-a055-0d27255a9b68 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:22.602969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:22.604459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:22.605889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:22.606326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:22.610835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.615644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 854/1727] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:22.682962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:22.686278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:22.777916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:22.821549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:22.823173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464fa95-1912-479f-bf3d-6146a3c7405d service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:22.853142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 874/1728] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:22.914152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0db98-259b-4d5b-9ba0-86fb14915bba None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:22.925972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:22.927851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:22.928167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:22.928543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:22.976021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:22.978642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:23.023951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 855/1729] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:23.493452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:23.494054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:23.494619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:23.525256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:23.527541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:23.555319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:23.563647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:23.587080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f748830-8476-4ce4-bbe8-d828957982f2 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:23.597402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 875/1730] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Jun 30 04:15:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:23.600885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:23.615406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:23.616530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:23.617460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:23.617919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:23.618192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] PATH_INFO: `/v3/users/fea8685db2144c208afe1c9de7c9af80` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:23.621215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea8685db2144c208afe1c9de7c9af80)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:23.633404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:23.641170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:23.670354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76442bf-3204-4130-b38a-95ce95d91a56 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:23.703174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:23.704924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:23.706758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:23.710992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] PATH_INFO: `/v3/users/cfaf12d12d6a423bb97dd5353e98bf67` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:23.713665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfaf12d12d6a423bb97dd5353e98bf67)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:23.719757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:23.768702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea8685db2144c208afe1c9de7c9af80'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.830084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea8685db2144c208afe1c9de7c9af80'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.891677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfaf12d12d6a423bb97dd5353e98bf67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.942804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea8685db2144c208afe1c9de7c9af80'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.944499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea8685db2144c208afe1c9de7c9af80'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.959737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfaf12d12d6a423bb97dd5353e98bf67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.977976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea8685db2144c208afe1c9de7c9af80'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.995704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfaf12d12d6a423bb97dd5353e98bf67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.996454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfaf12d12d6a423bb97dd5353e98bf67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:23.997267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfaf12d12d6a423bb97dd5353e98bf67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.005101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 856/1731] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:23 2022] DELETE /identity/v3/users/fea8685db2144c208afe1c9de7c9af80 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:24.006540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.008656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.037797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 876/1732] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:23 2022] DELETE /identity/v3/users/cfaf12d12d6a423bb97dd5353e98bf67 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:24.038710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.042765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.196280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f99f1-c4d1-46dd-970a-72ea776b8fe7 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.241700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380faec-736f-47e7-9dff-1d5be6fe164d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.251256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.252788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.253234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.253726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.273747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.290956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:24.293614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bde828-2536-443e-bed3-be9bfa6f645a service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:24.298588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.299606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.300062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.300388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.309251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 877/1733] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:24.311395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.311904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.312312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:24.312572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.313858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d6bc4-9f7b-4442-9474-6b9e2248edfe service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:24.329949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 857/1734] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:24.331574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9631-035a-4998-a07d-b17a8116291e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.331872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9631-035a-4998-a07d-b17a8116291e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.332165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d9631-035a-4998-a07d-b17a8116291e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.342465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:24.345287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.376585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-436d9631-035a-4998-a07d-b17a8116291e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:24.380882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-436d9631-035a-4998-a07d-b17a8116291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.427974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:24.467906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:24.501487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 878/1735] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:24.501487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.504356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.505331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 858/1736] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:24.507362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9631-035a-4998-a07d-b17a8116291e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.509765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-436d9631-035a-4998-a07d-b17a8116291e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.529036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fd2af-661b-45fb-88ef-e560682846c6 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.531759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d9631-035a-4998-a07d-b17a8116291e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.544035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.545928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.547812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.548471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] PATH_INFO: `/v3/projects/1b2cfea312a04240a0a0e709879d4832` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.549556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.551197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.551601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2cfea312a04240a0a0e709879d4832)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:24.551718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.552035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] PATH_INFO: `/v3/users/3c8661e8aa094e839426aca1c5c5db62` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.554890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8661e8aa094e839426aca1c5c5db62)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:24.556377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:24.561788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:24.641161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2cfea312a04240a0a0e709879d4832', 'request_id': 'req-20404343-c1e8-4808-8dc7-1d190defcec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.656700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2cfea312a04240a0a0e709879d4832', 'request_id': 'req-20404343-c1e8-4808-8dc7-1d190defcec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.676002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2cfea312a04240a0a0e709879d4832', 'request_id': 'req-20404343-c1e8-4808-8dc7-1d190defcec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.695980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] The token cache is being invalidate because project 1b2cfea312a04240a0a0e709879d4832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:24.697203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.700031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.703227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8661e8aa094e839426aca1c5c5db62'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.703835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 859/1737] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:24 2022] DELETE /identity/v3/projects/1b2cfea312a04240a0a0e709879d4832 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:24.705275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.707865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.741203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8661e8aa094e839426aca1c5c5db62'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.754766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8661e8aa094e839426aca1c5c5db62'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.756953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8661e8aa094e839426aca1c5c5db62'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.758825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8661e8aa094e839426aca1c5c5db62'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:24.786834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 879/1738] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:24 2022] DELETE /identity/v3/users/3c8661e8aa094e839426aca1c5c5db62 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:24.787768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:24.794067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.826565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20404343-c1e8-4808-8dc7-1d190defcec3 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.939481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a7cbd-3dad-48dd-a1a3-16d58b689071 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:24.943173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.944011 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.944323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.944743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.947573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:24.949780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:24.949780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:24.951428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:24.954512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:24.954642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.016398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:25.092172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.096227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39151cd4-876e-48f5-b446-6e6dfb37a139 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.114900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 860/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:25.116857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9158f3-c5cb-4983-ab61-f0eb5b23c172 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.117326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9158f3-c5cb-4983-ab61-f0eb5b23c172 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.117769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9158f3-c5cb-4983-ab61-f0eb5b23c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.127287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.138087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252175d2-e404-4c13-9910-47b0ab19766a service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.145041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-db9158f3-c5cb-4983-ab61-f0eb5b23c172 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:25.146667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-db9158f3-c5cb-4983-ab61-f0eb5b23c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.169192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 880/1740] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:25.174055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.180763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 861/1741] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:25.181484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.181916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.190425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.190718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.191012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.222040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:25.235360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.239606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:25.240968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.255929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 881/1742] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:25.256105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 862/1743] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:25.259419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.274731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc108e-aa42-40eb-93af-59928062ac78 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:25.276784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.278932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.299500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:25.302066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c736ca-1af3-4d96-8da4-5e581c4e2577 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:25.302263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.302633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.302919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.304595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:25.304924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.307116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.312368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:25.314331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:25.315175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.315663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.315663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.353918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.357412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.358144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 882/1744] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:25.362939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.402137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba898965-71a7-49f8-87f2-e8a060702008 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:25.404306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 863/1745] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:25.407168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.411702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.419946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:25.423041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.423394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.423696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.424475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.427056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.456440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5e3e5-ada0-41b7-af08-d5051947793b service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:25.468624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:25.472156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.472930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.473218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] PATH_INFO: `/v3/projects/c9be5d4573e345fc9c1e4a065ed94c9a` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.476260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9be5d4573e345fc9c1e4a065ed94c9a)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.480600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.501766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 883/1746] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:25.506540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.657402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9be5d4573e345fc9c1e4a065ed94c9a', 'request_id': 'req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:25.673992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9be5d4573e345fc9c1e4a065ed94c9a', 'request_id': 'req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:25.686513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9be5d4573e345fc9c1e4a065ed94c9a', 'request_id': 'req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:25.716157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] The token cache is being invalidate because project c9be5d4573e345fc9c1e4a065ed94c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:25.719023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:25.720841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a80c87ac-9bea-4d64-aa7b-cc8a07dea623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:25.725496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 864/1747] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:25 2022] DELETE /identity/v3/projects/c9be5d4573e345fc9c1e4a065ed94c9a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:25.726171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81be00-549e-4f51-a848-5325eda11b30 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:25.730562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203ce8-7397-4351-bd0c-531e4ecda1f7 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.731405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203ce8-7397-4351-bd0c-531e4ecda1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.731692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203ce8-7397-4351-bd0c-531e4ecda1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.797192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-07203ce8-7397-4351-bd0c-531e4ecda1f7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:25.801399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-07203ce8-7397-4351-bd0c-531e4ecda1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:25.897474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:25.900449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:25.900688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:25.901164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:25.903216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:25.908387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:25.976600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 884/1748] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/users => generated 288 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:25.980361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:25.980917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 865/1749] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:25.990566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:26.023864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.025626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.026411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.042659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927145a3-2644-4727-bcd1-02d38e10a5ee admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.071567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.073390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.081019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:26.082811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:26.089010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.089010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.089764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.091484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.099312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 866/1750] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:26.103737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.128730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 885/1751] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:26.130779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.132761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09760eaa-ce7e-4cc0-87bb-428b9b3e03f6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.139715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.140399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.140745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.141027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.141836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:26.142145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.150544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.153501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af27da-a5f8-4b29-8d4c-7148fb3cd312 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.170797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.174180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.175153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 867/1752] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:26.177715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.178078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.178878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.184247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.194649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.206767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:26.213126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670a7d7-f015-4034-bca3-e090e28e4fca admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.215812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 886/1753] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:26.222678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.226855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.228017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.229165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.229765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.230947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.233678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.248180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a528e-fafd-4ce3-b11a-fadba0ec56c1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.257239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.258233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.258741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.259046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] PATH_INFO: `/v3/projects/f3525c22b215472ca04b6cd2a7e76f84/users/bc92263566814e9b98d611d0c03f5e81/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.296514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3525c22b215472ca04b6cd2a7e76f84, user_id=bc92263566814e9b98d611d0c03f5e81, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.299256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 868/1754] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:26 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:26.303355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426a2ba-35d3-4bd3-ab03-753f9673c13c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.313569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.329433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 887/1755] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:26 2022] PUT /identity/v3/projects/f3525c22b215472ca04b6cd2a7e76f84/users/bc92263566814e9b98d611d0c03f5e81/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:26.341715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.343724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.344512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.439699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] MFA Rules not processed for user `bc92263566814e9b98d611d0c03f5e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:26.460571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:26.523696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b60ae-8524-43df-a6d4-34a21933bd6f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.527438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:26.719534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.722085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.723395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.724631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.727246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.735870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.793406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 869/1756] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:26 2022] POST /identity/v3/users => generated 282 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:26.797869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.836459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 888/1757] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 497 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:26.852935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b80b39-e19d-4cd0-891a-922e913aba55 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.888634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.892422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.892720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.893003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.894850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.896619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:26.916096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 870/1758] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:26.917234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.928787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:26.939080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:26.943934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c5240-1bfb-4007-b83d-e7409e121c5f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:26.958620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:26.962125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:26.962125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:26.962125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:26.962807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:26.967804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:27.002086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 889/1759] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:27.003125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:27.026399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cfdea-d2f4-48ad-9e03-52246939277d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:27.040091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:27.041080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.041398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.041656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] PATH_INFO: `/v3/projects/1c912ba174884ddcbae4013f58cc04af/users/8c892b66af75490f981694e73e8d9acd/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.078606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c912ba174884ddcbae4013f58cc04af, user_id=8c892b66af75490f981694e73e8d9acd, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:27.084589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50fbef-dd3d-4c69-a3c1-f8722c5cd681 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:27.137562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 890/1760] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:27 2022] PUT /identity/v3/projects/1c912ba174884ddcbae4013f58cc04af/users/8c892b66af75490f981694e73e8d9acd/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:27.141765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c645-0620-409e-b5a9-d74d13ec40b7 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.142054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c645-0620-409e-b5a9-d74d13ec40b7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.142331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c645-0620-409e-b5a9-d74d13ec40b7 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.153590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 891/1761] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:15:27 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:27.155176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.155470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.155767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.172590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 892/1762] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:15:27 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:27.173538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:27.179927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:27.236515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837e43b-7cab-41dc-866d-d424e68ea1c2 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:27.253761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:27.426346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d105915-2bef-459a-a554-9ce0277d2c3d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:27.434162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:27.435367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.435852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.436139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] PATH_INFO: `/v3/projects/14c8ea32df0041e89f6a8dad4007d7b8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.437495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c8ea32df0041e89f6a8dad4007d7b8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:27.442406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:27.443025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.443348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.443627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.450064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:27.589603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c8ea32df0041e89f6a8dad4007d7b8', 'request_id': 'req-607a15b0-3fd4-43d2-81c3-8617e39e65b4'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:27.610233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c8ea32df0041e89f6a8dad4007d7b8', 'request_id': 'req-607a15b0-3fd4-43d2-81c3-8617e39e65b4'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:27.625459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c8ea32df0041e89f6a8dad4007d7b8', 'request_id': 'req-607a15b0-3fd4-43d2-81c3-8617e39e65b4'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:27.654111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] The token cache is being invalidate because project 14c8ea32df0041e89f6a8dad4007d7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:27.654998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:27.655973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-607a15b0-3fd4-43d2-81c3-8617e39e65b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:27.661527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 893/1763] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:27 2022] DELETE /identity/v3/projects/14c8ea32df0041e89f6a8dad4007d7b8 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:27.663577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e0552-eeff-4bd1-b824-485a81dca392 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.663968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e0552-eeff-4bd1-b824-485a81dca392 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.664541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e0552-eeff-4bd1-b824-485a81dca392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.692715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:27.695255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:27.699733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed9a329-952b-4fa3-ac0b-469a0cd659d4 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:27.725867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c10e0552-eeff-4bd1-b824-485a81dca392 None None] MFA Rules not processed for user `8c892b66af75490f981694e73e8d9acd`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:27.731874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c10e0552-eeff-4bd1-b824-485a81dca392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:27.736627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 871/1764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:27.738249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.738496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.738803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:27.800763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:27.802167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:27.952347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 894/1765] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:27.956147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:27.956434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:27.956715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.042105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:28.046981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.050040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 872/1766] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:28.051078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.053012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.085772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 895/1767] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:28.088593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.093452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.154641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:28.216426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb6140-8327-4cf9-8ba1-61591598127f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:28.221876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:28.222879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:28.223371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:28.223662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] PATH_INFO: `/v3/users/94040d6963e946459856a3217c98e3d8` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.224993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=94040d6963e946459856a3217c98e3d8)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:28.228025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:28.288366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73462fbc-6241-4777-bd51-3b578f1ade4c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:28.301111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:28.302106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:28.302397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:28.302681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] PATH_INFO: `/v3/projects/ddd0bdde782b44a8957ab7b8ab2d38bf` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.307564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd0bdde782b44a8957ab7b8ab2d38bf)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:28.312809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:28.354307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94040d6963e946459856a3217c98e3d8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.376934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94040d6963e946459856a3217c98e3d8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.403346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94040d6963e946459856a3217c98e3d8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.404718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94040d6963e946459856a3217c98e3d8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.419063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94040d6963e946459856a3217c98e3d8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.436990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 873/1768] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:28 2022] DELETE /identity/v3/users/94040d6963e946459856a3217c98e3d8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:28.437707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.452122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.479761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd0bdde782b44a8957ab7b8ab2d38bf', 'request_id': 'req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.499937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd0bdde782b44a8957ab7b8ab2d38bf', 'request_id': 'req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.511192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd0bdde782b44a8957ab7b8ab2d38bf', 'request_id': 'req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.558951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec48ad-9f1b-4447-9aa4-e901e71faf0b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:28.570447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:28.573571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:28.573862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:28.574149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] PATH_INFO: `/v3/projects/f70ad89203de4b5cb4853fa4d0d6eeb4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.574940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] The token cache is being invalidate because project ddd0bdde782b44a8957ab7b8ab2d38bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:28.576843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70ad89203de4b5cb4853fa4d0d6eeb4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:28.578321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.581283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.581810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:28.584738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 896/1769] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:28 2022] DELETE /identity/v3/projects/ddd0bdde782b44a8957ab7b8ab2d38bf => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:28.588037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.592684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.669549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:28.747220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70ad89203de4b5cb4853fa4d0d6eeb4', 'request_id': 'req-78d46152-2148-4cb1-894d-b8edf3146c1f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.763925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84221dde-0571-4db0-9ec1-dc9b24e4d7b6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:28.772377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70ad89203de4b5cb4853fa4d0d6eeb4', 'request_id': 'req-78d46152-2148-4cb1-894d-b8edf3146c1f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.774492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:28.785005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70ad89203de4b5cb4853fa4d0d6eeb4', 'request_id': 'req-78d46152-2148-4cb1-894d-b8edf3146c1f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.832725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] The token cache is being invalidate because project f70ad89203de4b5cb4853fa4d0d6eeb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:28.833981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.836174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:28.843213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 874/1770] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:28 2022] DELETE /identity/v3/projects/f70ad89203de4b5cb4853fa4d0d6eeb4 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:28.847987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.850464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:28.947284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:28.947986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:28.948272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:28.948556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.949402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:28.951592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:28.973348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d46152-2148-4cb1-894d-b8edf3146c1f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:28.979980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:28.981090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:28.981421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:28.981745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] PATH_INFO: `/v3/users/5b6e7e9c34084a569a1cf0609c2800e6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:28.983827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b6e7e9c34084a569a1cf0609c2800e6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:28.988616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 897/1771] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:28 2022] POST /identity/v3/projects => generated 363 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:28.988799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:28.988893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:28.990333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.073376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b6e7e9c34084a569a1cf0609c2800e6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.089432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b6e7e9c34084a569a1cf0609c2800e6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.106625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b6e7e9c34084a569a1cf0609c2800e6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.107626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b6e7e9c34084a569a1cf0609c2800e6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.129170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08ce99-17d4-4b88-bb3f-0f0d482dbd47 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.130070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b6e7e9c34084a569a1cf0609c2800e6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.135144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.136992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.136992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.136992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] PATH_INFO: `/v3/users/d78f642f27ad4162a496d56773d87219` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.138793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78f642f27ad4162a496d56773d87219)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.149933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.150975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 875/1772] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:28 2022] DELETE /identity/v3/users/5b6e7e9c34084a569a1cf0609c2800e6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:29.155580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.157921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.300214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:29.308644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2464bfd0-7b4a-4adb-ae96-5a1025a92eb8 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.324041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.325816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.325949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.326201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.328629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78f642f27ad4162a496d56773d87219'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.333599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.340963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:29.350855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78f642f27ad4162a496d56773d87219'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.371180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.375072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.386523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78f642f27ad4162a496d56773d87219'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.387308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78f642f27ad4162a496d56773d87219'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.389855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78f642f27ad4162a496d56773d87219'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.391647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 876/1773] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:29.392519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.394050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.414839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa6c5d-5520-499b-8e12-5395a6dee15a service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.423646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.423646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.423646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.423646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.424251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.445105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.446740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.455180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 898/1774] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:28 2022] DELETE /identity/v3/users/d78f642f27ad4162a496d56773d87219 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:29.456460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.459551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.481375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 877/1775] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:29.482663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.486555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.632363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8625172-983d-4a97-bf9d-78f1e788df9e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.644311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.646969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.647509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.648224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.649125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.652386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.679249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8db6d-9198-4d3e-9e75-2bfe33965c62 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.684539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.685652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.686106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.686452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] PATH_INFO: `/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.687769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c813fd6842e14a4cb2ee595c9512c7d6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.690064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.694937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 899/1776] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:29 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:29.701186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.722945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efedf6-5bbd-45b3-970c-6db8425c28af admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.737755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.738565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.740339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.740629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:29.741388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:29.747182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:29.786352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 900/1777] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:29.788417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.797415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c813fd6842e14a4cb2ee595c9512c7d6', 'request_id': 'req-58eaf8c0-f414-42f8-8041-621cb62d0d32'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.816930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44433216-f31a-4ca9-8def-5c5b918bb0af admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:29.836949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c813fd6842e14a4cb2ee595c9512c7d6', 'request_id': 'req-58eaf8c0-f414-42f8-8041-621cb62d0d32'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.846652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c813fd6842e14a4cb2ee595c9512c7d6', 'request_id': 'req-58eaf8c0-f414-42f8-8041-621cb62d0d32'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.879948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] The token cache is being invalidate because project c813fd6842e14a4cb2ee595c9512c7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:29.880834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.881834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:29.883891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 878/1778] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:29 2022] DELETE /identity/v3/projects/c813fd6842e14a4cb2ee595c9512c7d6 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:29.884932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:29.889780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:29.994833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:29.995820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:29.996180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:29.996355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] PATH_INFO: `/v3/projects/43a403ce7beb4cf28ca1296d9579eace/users/23754d77014a41cbb62d7c81daaac599/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.040555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a403ce7beb4cf28ca1296d9579eace, user_id=23754d77014a41cbb62d7c81daaac599, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.043200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f0b81-937b-4cdd-a75f-47991043e923 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.062718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 901/1779] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:29 2022] PUT /identity/v3/projects/43a403ce7beb4cf28ca1296d9579eace/users/23754d77014a41cbb62d7c81daaac599/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:30.064706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.065807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.066248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.092533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:30.094037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.104945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eaf8c0-f414-42f8-8041-621cb62d0d32 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:30.126285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:30.172462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:30.213801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:30.215602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.216097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.216528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] PATH_INFO: `/v3/users/bc92263566814e9b98d611d0c03f5e81` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.218233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc92263566814e9b98d611d0c03f5e81)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.220725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.265048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 902/1780] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:30.266144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:30.269460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.300080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f40d42-7e12-44f7-9745-3cdc21e12ad4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:30.312427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:30.314304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.314609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.314900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.315689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.319238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc33625-c33e-4138-8e08-4295e262d50f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.334581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc92263566814e9b98d611d0c03f5e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:30.350667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 903/1781] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:30.355504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc92263566814e9b98d611d0c03f5e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:30.359472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df76e4-0d42-4081-a3e0-3f611bbdc410 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.359770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df76e4-0d42-4081-a3e0-3f611bbdc410 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.360043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df76e4-0d42-4081-a3e0-3f611bbdc410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.386900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc92263566814e9b98d611d0c03f5e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:30.388415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc92263566814e9b98d611d0c03f5e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:30.396752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e9df76e4-0d42-4081-a3e0-3f611bbdc410 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:30.398676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e9df76e4-0d42-4081-a3e0-3f611bbdc410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.402808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc92263566814e9b98d611d0c03f5e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:30.414862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 879/1782] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:29 2022] DELETE /identity/v3/users/bc92263566814e9b98d611d0c03f5e81 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:30.418333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:30.421986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.423467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 904/1783] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:30.426644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.427803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.429685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.470993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:30.476144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.695603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 905/1784] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:30.704573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:30.726297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed14060-e3e7-45ce-bdd7-cf947c9e3698 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:30.744441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:30.745193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.746205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.746596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.748020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:30.748351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.750036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.778565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 906/1785] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:30.779871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:30.782524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:30.821414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1806a9-3218-42a5-9c09-7f6027d8f6cd admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:30.835355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66412d5-a0d7-4f25-9fe0-98be2c1aa306 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:30.855107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:30.856153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.856474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.856767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] PATH_INFO: `/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.867846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9734d3f95b9f48aaa7f96c47fad4cd98)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.870261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.871478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:30.872631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:30.873415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:30.873749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] PATH_INFO: `/v3/projects/43a403ce7beb4cf28ca1296d9579eace/users/23754d77014a41cbb62d7c81daaac599/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:30.881360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a403ce7beb4cf28ca1296d9579eace, user_id=23754d77014a41cbb62d7c81daaac599, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:30.885026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:30.939917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 880/1786] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:30 2022] PUT /identity/v3/projects/43a403ce7beb4cf28ca1296d9579eace/users/23754d77014a41cbb62d7c81daaac599/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:30.940981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:30.964046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.022441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9734d3f95b9f48aaa7f96c47fad4cd98', 'request_id': 'req-b7d1b09b-faee-441f-b07c-4cb676fa8d82'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.047824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9734d3f95b9f48aaa7f96c47fad4cd98', 'request_id': 'req-b7d1b09b-faee-441f-b07c-4cb676fa8d82'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.054071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:31.063087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9734d3f95b9f48aaa7f96c47fad4cd98', 'request_id': 'req-b7d1b09b-faee-441f-b07c-4cb676fa8d82'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.087548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] The token cache is being invalidate because project 9734d3f95b9f48aaa7f96c47fad4cd98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:31.088089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.088870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.090881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 907/1787] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:30 2022] DELETE /identity/v3/projects/9734d3f95b9f48aaa7f96c47fad4cd98 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:31.092090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:31.098399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.103145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369e9a5-66d3-4831-9db3-110ef5d7e525 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.160076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:31.187726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.188645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.189032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.189659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.193363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.233279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1b09b-faee-441f-b07c-4cb676fa8d82 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.238244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.239018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.239245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.239518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.240214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:31.242118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd0e9f-d175-4be6-9587-7e5a2f89e15f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:31.293123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 908/1788] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/projects => generated 409 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:31.294434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2f63-17ac-4a2d-9b6c-e69fb6109e7f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.294752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2f63-17ac-4a2d-9b6c-e69fb6109e7f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.295168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d2f63-17ac-4a2d-9b6c-e69fb6109e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.301171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:31.302583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:31.315424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 881/1789] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:31.316420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:31.317826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.343297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5957c9-f5f7-4b03-b037-52e3dc6c1ce0 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.345945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1f5d2f63-17ac-4a2d-9b6c-e69fb6109e7f None None] MFA Rules not processed for user `23754d77014a41cbb62d7c81daaac599`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:31.347392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1f5d2f63-17ac-4a2d-9b6c-e69fb6109e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.477071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.478062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.478358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.478843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] PATH_INFO: `/v3/projects/38fc0a32fc9145898385dab18b623948` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.480003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38fc0a32fc9145898385dab18b623948)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:31.483675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:31.511393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 909/1790] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:31.512997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.513160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.518969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.555219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:31.556488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.588841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 910/1791] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:31.588841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:31.610819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f867be8-9e16-4c33-822f-381db1b08cee None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.621886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:31.641505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38fc0a32fc9145898385dab18b623948', 'request_id': 'req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.671011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38fc0a32fc9145898385dab18b623948', 'request_id': 'req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.681088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38fc0a32fc9145898385dab18b623948', 'request_id': 'req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.703492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] The token cache is being invalidate because project 38fc0a32fc9145898385dab18b623948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:31.703965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.704897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:31.711007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 882/1792] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:31 2022] DELETE /identity/v3/projects/38fc0a32fc9145898385dab18b623948 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:31.712708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:31.715299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.731480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.732660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.732660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.733199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.733629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:31.735597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654823a4-6dd2-406e-acf5-dbe8b3bce5ee admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:31.790020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 911/1793] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/users => generated 305 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:31.791175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c194-2e36-418c-abf2-e20f9cfcaa37 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.791477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c194-2e36-418c-abf2-e20f9cfcaa37 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.791761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4c194-2e36-418c-abf2-e20f9cfcaa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.837583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-aac4c194-2e36-418c-abf2-e20f9cfcaa37 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:31.838765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aac4c194-2e36-418c-abf2-e20f9cfcaa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.845362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dfb43-e2e1-4643-840a-f4e6d38c34d1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.850775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.852367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.853085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.853463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.856322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.857856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 912/1794] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:31.860605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.860605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.860605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.898600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:31.900761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.912316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 913/1795] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:31.913380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:31.920308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:31.933386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b54588-2b10-43bd-8d42-4cd820989a41 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:31.954633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:31.955411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:31.955787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:31.956775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:31.959140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:31.976214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:31.979811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:32.000658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 914/1796] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:32.003276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.003566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.005396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.006133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.025284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972f7c1-6775-4196-bbad-ef4e7af9654e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.036584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 883/1797] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:32.037168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.037423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.038481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.039129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.039496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] PATH_INFO: `/v3/projects/f3525c22b215472ca04b6cd2a7e76f84` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.041101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3525c22b215472ca04b6cd2a7e76f84)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.044090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.059601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6275db-33d0-4995-82a6-6dc4fbc6c673 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.070972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.071692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.071983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.072400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.073337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:32.073723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.082513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38d947-f0f7-4daa-86d5-3c33b88c0a1f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.107375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 884/1798] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:32.107375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.108033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.108198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.144353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3525c22b215472ca04b6cd2a7e76f84', 'request_id': 'req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:32.157980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:32.159348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.162560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3525c22b215472ca04b6cd2a7e76f84', 'request_id': 'req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:32.189299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3525c22b215472ca04b6cd2a7e76f84', 'request_id': 'req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:32.212386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] The token cache is being invalidate because project f3525c22b215472ca04b6cd2a7e76f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:32.212873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:32.214529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:32.222471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 915/1799] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:32 2022] DELETE /identity/v3/projects/f3525c22b215472ca04b6cd2a7e76f84 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:32.223857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.225457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.278735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:32.298697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 885/1800] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:32.299774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.301321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.329143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97653a42-65de-4b0c-b74e-d22a0ecdc342 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.341152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.342039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.342336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.342658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.343452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.345586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.417078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 886/1801] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:32.418094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.421083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.434721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b9941-3ad7-4fb1-83e4-9abba67c6fba admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.524998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90de2a-5bd2-48d6-ba7a-6e8d44ed79c0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.530145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.530880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.531198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.531479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.534248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.555162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.556720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.557288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.558153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.559209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.562266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bd172-f408-4377-9f11-db4138939a27 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.582573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 916/1802] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:32.587338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.587854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.588426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.622567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:32.624030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.641725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 917/1803] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:32.642973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.654201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734c619-1a12-48c7-9406-b785ccd935e8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.663141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.663141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.663141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.663141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.663565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:32.663565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.664118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.680772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 918/1804] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:32.681747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.693633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:32.698574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6bf8fc-71d6-40dd-8100-45a22b73212a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.703179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.706058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.715829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.718375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.718997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.719632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.720428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 887/1805] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:32.721379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.724428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.728065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f84d5b-0f4c-4666-98ed-2adc0d631224 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.740165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ea4de-8066-4d31-8acd-f8c6240e1d28 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.753259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.755444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.756028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.757374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] PATH_INFO: `/v3/projects/9953e3ce880a4f1fbc900eb606d315ee/users/5e09e89dd40146debeb07218ec466fcd/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.782449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 919/1806] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:32.784388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.784875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.785341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.805119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9953e3ce880a4f1fbc900eb606d315ee, user_id=5e09e89dd40146debeb07218ec466fcd, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:32.808274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:32.811566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:32.814091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:32.835601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 920/1807] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:32.836852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.839186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 888/1808] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:32 2022] PUT /identity/v3/projects/9953e3ce880a4f1fbc900eb606d315ee/users/5e09e89dd40146debeb07218ec466fcd/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:32.840222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:32.849646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a678f-3628-4d26-babb-6ae36b19de9c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:32.992601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:32.996574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:32.996574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:32.996985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:32.998444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.002745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.015681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513004c-3e94-456e-9395-666698daff91 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.032502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.033809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.034425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.034964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.036641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.038743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ca18c-2370-49e4-b14a-8c97d83c1219 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.057126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 921/1809] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:32 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.058419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.064555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 889/1810] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.066235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.066596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.067060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.077463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:33.107155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] MFA Rules not processed for user `5e09e89dd40146debeb07218ec466fcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:33.113269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:33.180302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e1054-c9a3-4923-b0c8-ad91444bc172 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.187859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.188566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.188880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.189139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.189896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:33.190238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.191917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.210457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 922/1811] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:33.210457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.222796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08b440-59a0-42d9-b8af-f740a8aa65cc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.227984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.228677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.228968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.229253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.229498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 890/1812] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:33.230169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.230357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.232416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.242654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41629ad3-4028-40ec-95e5-b10e5d055de6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.248646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.250603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.250986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.251433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.252237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.254154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.261331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 923/1813] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/users => generated 287 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.261331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.269752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 891/1814] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.269752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.277446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f1163-fb84-4bae-a8b5-9655bbf3c6e1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.283163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.283853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.284144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.284418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.285118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.286295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9a45-72d2-4fd1-8452-c58bd0230f1e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.287417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.293596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.294345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.294691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.295312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.296201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.299368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.319593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 924/1815] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.320898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.335275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 892/1816] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/projects => generated 407 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.336114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.348913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:33.393854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8eff39-0fa9-4100-8cf6-51161a17ac0e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.407458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe19824-8802-49cd-ad83-6de34e85bbdb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.412386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.413297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.413586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.413885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] PATH_INFO: `/v3/projects/58cb41b3cc0b44339f82fbb494180985/users/3bf5a65cfddf4fe19ed8ec7073b8c2c9/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.429473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58cb41b3cc0b44339f82fbb494180985, user_id=3bf5a65cfddf4fe19ed8ec7073b8c2c9, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.432325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.446187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 893/1817] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:33 2022] PUT /identity/v3/projects/58cb41b3cc0b44339f82fbb494180985/users/3bf5a65cfddf4fe19ed8ec7073b8c2c9/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:33.446974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.448560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.449244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.449527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.449807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.450588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.452280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.463013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 925/1818] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.463870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.475289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7f752-479a-4dfe-8edb-3123e7fe9f0b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.508322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:33.527599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e230cd4-8e55-4bf0-a91b-7759fde9f3e3 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.534427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.535149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.535966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.536251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.537419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.539435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b797de-376c-42f4-bab1-7a90d7935de4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.548934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.549628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.550232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.550464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.551216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.553138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.575195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 894/1819] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/users => generated 281 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.576954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.577248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.577523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.595816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 926/1820] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/users => generated 310 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:33.595816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.611888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09108a0-c15a-440b-93df-2ee8730e314e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.618937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.619664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.619942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.620215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.621255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.625236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.643036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 927/1821] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.644145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.654079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] MFA Rules not processed for user `3bf5a65cfddf4fe19ed8ec7073b8c2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:33.655970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:33.656117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220a33e-9998-44ac-8b2b-eed560f70ec0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.663150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.663900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.664190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.664453 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.665207 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.666967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.688490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 928/1822] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.689392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.702502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99df7d-e6fe-47ef-a6fa-706d76e0897f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.712793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.713527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.713841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.714114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.714957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.716767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.729255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 929/1823] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.730455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.741848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc98f86-2249-4dd8-993b-d9bf9b7c1ee8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.746665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.747648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.748069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.748564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] PATH_INFO: `/v3/projects/6c4f5e5f79414af1ac4b7237ac1fea11/users/efa9c6f5ee034e11a7943fbccd43dec2/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.755278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 895/1824] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:33.756152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.769449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c4f5e5f79414af1ac4b7237ac1fea11, user_id=efa9c6f5ee034e11a7943fbccd43dec2, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.769932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c0471-9922-47b8-b607-162579f81ade None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.774038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.775056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.775836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.776122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.776394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.777129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.778663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:33.778778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.793184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 896/1825] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:33.793909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.794390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 930/1826] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:33 2022] PUT /identity/v3/projects/6c4f5e5f79414af1ac4b7237ac1fea11/users/efa9c6f5ee034e11a7943fbccd43dec2/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:33.795306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.797048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:33.809911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f0e3f-3eec-4d90-ae1a-21b845a9fdd0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.856252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f16afb-8c67-44b6-8f5e-3b75c864d098 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.860147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.860806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.861091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.861366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.868812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:33.869716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.869996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.870271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] PATH_INFO: `/v3/projects/9c2442c39eca424eb256fb301065b178/users/7c011733f3f44b06a2644d8d94f67d98/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.885065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2442c39eca424eb256fb301065b178, user_id=7c011733f3f44b06a2644d8d94f67d98, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.887837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456a97ef-2951-463a-af77-0fe0b658cdd9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.901649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 897/1827] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:33 2022] PUT /identity/v3/projects/9c2442c39eca424eb256fb301065b178/users/7c011733f3f44b06a2644d8d94f67d98/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:33.903010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:33.903399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:33.903741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:33.917386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:33.918833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:33.927841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 931/1828] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:33.929650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] MFA Rules not processed for user `efa9c6f5ee034e11a7943fbccd43dec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:33.930416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:33.930819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:33.941768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ab618-04e6-4eaf-a223-71a5ad949529 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:33.956696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:34.004511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 898/1829] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:34.005452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.038082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.038951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.039341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.039559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] PATH_INFO: `/v3/projects/8e28a82346e34d01b702bad4673cc11a/users/bfb0d71121ee48c38ec0507f4340244e/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.058213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e28a82346e34d01b702bad4673cc11a, user_id=bfb0d71121ee48c38ec0507f4340244e, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.061723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.078600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 932/1830] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:33 2022] PUT /identity/v3/projects/8e28a82346e34d01b702bad4673cc11a/users/bfb0d71121ee48c38ec0507f4340244e/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:34.083355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.096820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc288-d294-4509-b8d0-7ca085a204e9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.144024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:34.167602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.168513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.168867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.169295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.170719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.172662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce265cc-748c-4e2f-8206-4742264671bf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.186045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 899/1831] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:34.187735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.187930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f225e87a-b2ee-41e9-b5a7-10d1c029df70 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.188147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.188493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.194143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.194860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.196176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.196176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.196818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.198890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.228347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] MFA Rules not processed for user `bfb0d71121ee48c38ec0507f4340244e`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:34.230252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:34.239241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 933/1832] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:34.240566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.319760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:34.332644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 900/1833] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:34.333637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.340254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d77e7-ba77-4222-86d9-44d9505ea098 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.347363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.348401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.348760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.349151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] PATH_INFO: `/v3/projects/9c2442c39eca424eb256fb301065b178/users/7c011733f3f44b06a2644d8d94f67d98/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.349796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdb85e-c352-49f6-9594-69581b0229db None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.359787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c2442c39eca424eb256fb301065b178, user_id=7c011733f3f44b06a2644d8d94f67d98, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.363044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.369644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.372131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.372472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.373154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.374384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.376543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1034de-7853-479e-97f7-d8e07e472031 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.377740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 934/1834] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:34 2022] PUT /identity/v3/projects/9c2442c39eca424eb256fb301065b178/users/7c011733f3f44b06a2644d8d94f67d98/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:34.378765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.403143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 901/1835] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:34.403524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.403721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.404174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.433090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] MFA Rules not processed for user `7c011733f3f44b06a2644d8d94f67d98`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:34.434342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:34.469588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede047fb-690f-4454-aef1-14bda5397656 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.475330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.476163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.476516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.476868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.477763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.479780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.509194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 935/1836] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:34.510247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.511234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 902/1837] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:34.514886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.525819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257473d-841b-4790-9e1b-5249f8e75f1a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.572266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:34.599551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2079-e163-44eb-a29f-692bf6bdf0a1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.603259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.603995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.604284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.604565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.604717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.605263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.605549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.605895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.606245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.607057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.607346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.609520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.620450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 936/1838] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:34.621559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.629294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 903/1839] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:34.630221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.640292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c71412-a84f-4d9a-b390-a62bc7c725af admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.645056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.646086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.646450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.646760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] PATH_INFO: `/v3/projects/ddf73e410f9e43a28d4798c41124bf1f/users/b9d11681fe9f4643adc39cc40ecbbc7e/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.663053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf73e410f9e43a28d4798c41124bf1f, user_id=b9d11681fe9f4643adc39cc40ecbbc7e, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.665990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.680136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 904/1840] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:34 2022] PUT /identity/v3/projects/ddf73e410f9e43a28d4798c41124bf1f/users/b9d11681fe9f4643adc39cc40ecbbc7e/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:34.680606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68d347-5780-42b7-a6dc-362477165a85 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.681009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.733463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:34.754143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.754981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.755458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.755960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.781540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39549138-9870-478c-b742-a7f976ad3802 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.786472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.787185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.787471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.787748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.788603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.790332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.802210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 905/1841] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:34.803247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.818720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5a13e-9d21-48b8-b13c-45570805dd92 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.824920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.825603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.826461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.826758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.829227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.831079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.841992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.843598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.848167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 906/1842] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:34.848167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.856480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aea1c84-95ca-4e84-9aab-d5a48955abfb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.856716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 937/1843] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:34.857689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.861613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.862514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.862795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.863069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] PATH_INFO: `/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a/users/f7521e8cc9f24099b28eef50fea28b1e/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.878310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19f3fb-4e8f-46ac-9d7d-3aa6e9b8b927 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:34.883562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a094f2211e745899ebdfb4d6ae65c2a, user_id=f7521e8cc9f24099b28eef50fea28b1e, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.886945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.906441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:34.907403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.907689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.907972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] PATH_INFO: `/v3/projects/ddf73e410f9e43a28d4798c41124bf1f/users/b9d11681fe9f4643adc39cc40ecbbc7e/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:34.910118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf73e410f9e43a28d4798c41124bf1f, user_id=b9d11681fe9f4643adc39cc40ecbbc7e, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:34.913483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79596d86-14a8-437b-9424-6dd17b948e0d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:34.932571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 907/1844] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:34 2022] PUT /identity/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a/users/f7521e8cc9f24099b28eef50fea28b1e/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:34.934154 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:34.947464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 938/1845] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:34 2022] PUT /identity/v3/projects/ddf73e410f9e43a28d4798c41124bf1f/users/b9d11681fe9f4643adc39cc40ecbbc7e/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:34.958753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:34.959136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:34.959489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.005026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] MFA Rules not processed for user `b9d11681fe9f4643adc39cc40ecbbc7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:35.006512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:35.031678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:35.045054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782094-4c9f-4185-9ae7-3a0dbc0c1584 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.114672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 939/1846] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:35.117052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:35.152295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.153126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.153601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.153713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.155139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.157108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:35.171376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 908/1847] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:35.175800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:35.189659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce18bb-1815-4a06-af55-15992472e44d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.197506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.198757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.199023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.199428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] PATH_INFO: `/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a/users/f7521e8cc9f24099b28eef50fea28b1e/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.202118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a094f2211e745899ebdfb4d6ae65c2a, user_id=f7521e8cc9f24099b28eef50fea28b1e, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.205099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:35.206521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07745c1-c9f9-4bd1-9b10-dda3efb3c00c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.210960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.211755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.212045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.212327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.219343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 909/1848] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:35 2022] PUT /identity/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a/users/f7521e8cc9f24099b28eef50fea28b1e/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:35.226837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:35.257052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:35.282166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.283680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d78986b-8d81-4df4-99eb-bf29688ec6a2 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:35.291147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3af43-459a-47b5-8c2f-e6ea48582a31 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.294903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.295043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 940/1849] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:35 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:35.298269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.298366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.298437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.298513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.298513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.298642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.328682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] MFA Rules not processed for user `f7521e8cc9f24099b28eef50fea28b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:35.330413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:35.373793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.375344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:35.386338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 910/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:35.391728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:35.413150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 941/1851] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:35 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:35.491527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:35.500611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f2131-b076-495b-b762-77a230e3a1b4 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.504694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.505351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.505640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.505920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.522296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.523807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e9b4a-1ad0-4b4e-9df8-3b7197d4dd5b service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:35.533858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 911/1852] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:35 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:35.875800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:35.877258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:35.971394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee4449-d0bb-4906-994d-0c6f09cdf689 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:35.988432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:35.989486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:35.989837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:35.990177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] PATH_INFO: `/v3/users/8c892b66af75490f981694e73e8d9acd` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:35.991734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c892b66af75490f981694e73e8d9acd)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:35.998121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.070089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c892b66af75490f981694e73e8d9acd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.088651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c892b66af75490f981694e73e8d9acd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.103519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c892b66af75490f981694e73e8d9acd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.104356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c892b66af75490f981694e73e8d9acd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.105126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c892b66af75490f981694e73e8d9acd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.126165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 942/1853] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:35 2022] DELETE /identity/v3/users/8c892b66af75490f981694e73e8d9acd => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:36.130493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.130791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.131077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.142210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.144466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.158706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:36.158706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.230884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515460d-7078-4f19-80a7-d6c3394bf2b4 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.231668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 912/1854] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:36.234904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.234904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.236817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.237804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.238031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.238307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] PATH_INFO: `/v3/users/efa9c6f5ee034e11a7943fbccd43dec2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.239571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] RBAC: Authorizing `identity:delete_user(user_id=efa9c6f5ee034e11a7943fbccd43dec2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.241547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.248321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bac54-838b-47fd-b79e-c04177fb122b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.253678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.254680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.254971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.255284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] PATH_INFO: `/v3/users/bfb0d71121ee48c38ec0507f4340244e` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.255935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:36.256927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb0d71121ee48c38ec0507f4340244e)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.258952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.308446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efa9c6f5ee034e11a7943fbccd43dec2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.311327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb0d71121ee48c38ec0507f4340244e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.317115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:36.321636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb0d71121ee48c38ec0507f4340244e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.323167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efa9c6f5ee034e11a7943fbccd43dec2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.331814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efa9c6f5ee034e11a7943fbccd43dec2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.332863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efa9c6f5ee034e11a7943fbccd43dec2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.333473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efa9c6f5ee034e11a7943fbccd43dec2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.336131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb0d71121ee48c38ec0507f4340244e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.336860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb0d71121ee48c38ec0507f4340244e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.345172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3642583-b2b3-40db-b999-3702c6cb1ce9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb0d71121ee48c38ec0507f4340244e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.348345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 943/1855] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:36 2022] DELETE /identity/v3/users/efa9c6f5ee034e11a7943fbccd43dec2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:36.349387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.350700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.354512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 913/1856] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:36 2022] DELETE /identity/v3/users/bfb0d71121ee48c38ec0507f4340244e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:36.358883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44902b3-482c-43fd-81a7-7c22f2afa507 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.358883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44902b3-482c-43fd-81a7-7c22f2afa507 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.358883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44902b3-482c-43fd-81a7-7c22f2afa507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.377903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d44902b3-482c-43fd-81a7-7c22f2afa507 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:36.379171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d44902b3-482c-43fd-81a7-7c22f2afa507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.411284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06477d-d079-4f65-b44f-6d3cb337101e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.441550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 914/1857] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:36.442925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.443255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.443537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.462279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:36.463547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.463712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.464396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.464677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.464991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.466866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.471899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 915/1858] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:36.472614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.477034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.485730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.487333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.487472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f8187-9e79-4cb6-b164-c7832317a74d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.491087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.491712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.491991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.492262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.497237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 944/1859] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:36.498108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.499473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.502049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.503528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.509425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26762d3-961b-4ed0-ba7c-ccf627217051 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.513287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.514057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.514429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.514623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.514852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 916/1860] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:36.524327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.525763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.538314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 945/1861] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:36.874852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.876338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:36.889120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1232-fcd5-4e8c-9d5d-56f8694ffd2e service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:36.893820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:36.894671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:36.894971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:36.895283 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] PATH_INFO: `/v3/projects/6c4f5e5f79414af1ac4b7237ac1fea11` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:36.896987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c4f5e5f79414af1ac4b7237ac1fea11)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:36.899131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:36.950951 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c4f5e5f79414af1ac4b7237ac1fea11', 'request_id': 'req-53017f36-85ef-452b-be9c-e714f14b5ad1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.958808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c4f5e5f79414af1ac4b7237ac1fea11', 'request_id': 'req-53017f36-85ef-452b-be9c-e714f14b5ad1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.964755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c4f5e5f79414af1ac4b7237ac1fea11', 'request_id': 'req-53017f36-85ef-452b-be9c-e714f14b5ad1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.976324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] The token cache is being invalidate because project 6c4f5e5f79414af1ac4b7237ac1fea11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:36.976611 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.977280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:36.978832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 917/1862] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:36 2022] DELETE /identity/v3/projects/6c4f5e5f79414af1ac4b7237ac1fea11 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:36.983050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:36.992700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c840fe2-196a-43c8-bb2d-063bbd9f9ff8 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.036019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:37.043442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.049324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.082069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.083340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.083784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.084215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] PATH_INFO: `/v3/users/b9d11681fe9f4643adc39cc40ecbbc7e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.085684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d11681fe9f4643adc39cc40ecbbc7e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.088435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.095649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:37.125068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53017f36-85ef-452b-be9c-e714f14b5ad1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.130928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.131763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.132162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.132586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] PATH_INFO: `/v3/projects/1c912ba174884ddcbae4013f58cc04af` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.133953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c912ba174884ddcbae4013f58cc04af)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.136137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.142904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d11681fe9f4643adc39cc40ecbbc7e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.158226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d11681fe9f4643adc39cc40ecbbc7e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.167437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d11681fe9f4643adc39cc40ecbbc7e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.168403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d11681fe9f4643adc39cc40ecbbc7e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.169123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d11681fe9f4643adc39cc40ecbbc7e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.184494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 946/1863] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:36 2022] DELETE /identity/v3/users/b9d11681fe9f4643adc39cc40ecbbc7e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.185770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.187390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.199960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c912ba174884ddcbae4013f58cc04af', 'request_id': 'req-5122dc54-6039-4b0c-b48c-b291d8e8b48c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.212474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c912ba174884ddcbae4013f58cc04af', 'request_id': 'req-5122dc54-6039-4b0c-b48c-b291d8e8b48c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.224577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c912ba174884ddcbae4013f58cc04af', 'request_id': 'req-5122dc54-6039-4b0c-b48c-b291d8e8b48c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.239407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] The token cache is being invalidate because project 1c912ba174884ddcbae4013f58cc04af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:37.239682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.240353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.245459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 918/1864] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/projects/1c912ba174884ddcbae4013f58cc04af => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.246991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.249329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.277359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f8e1-df49-44c9-97d3-0b856d30e3ce admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.284963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.286315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.286634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.286928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] PATH_INFO: `/v3/projects/8e28a82346e34d01b702bad4673cc11a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.288443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e28a82346e34d01b702bad4673cc11a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.290613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.351632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122dc54-6039-4b0c-b48c-b291d8e8b48c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.359899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e28a82346e34d01b702bad4673cc11a', 'request_id': 'req-97744d6c-0a9b-48b2-84ca-9fb500ae515d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.369794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e28a82346e34d01b702bad4673cc11a', 'request_id': 'req-97744d6c-0a9b-48b2-84ca-9fb500ae515d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.372034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:37.380171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e28a82346e34d01b702bad4673cc11a', 'request_id': 'req-97744d6c-0a9b-48b2-84ca-9fb500ae515d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.398186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] The token cache is being invalidate because project 8e28a82346e34d01b702bad4673cc11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:37.398760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.399680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.401596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 947/1865] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/projects/8e28a82346e34d01b702bad4673cc11a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.405372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.406853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.439560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.440496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.440787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.441346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] PATH_INFO: `/v3/users/23754d77014a41cbb62d7c81daaac599` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.442297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=23754d77014a41cbb62d7c81daaac599)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.444286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.460033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:37.478437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97744d6c-0a9b-48b2-84ca-9fb500ae515d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.484896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.486020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.486275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.486620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] PATH_INFO: `/v3/users/f7521e8cc9f24099b28eef50fea28b1e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.488120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7521e8cc9f24099b28eef50fea28b1e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.490262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.529276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23754d77014a41cbb62d7c81daaac599'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.540802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23754d77014a41cbb62d7c81daaac599'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.557197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23754d77014a41cbb62d7c81daaac599'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.558094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23754d77014a41cbb62d7c81daaac599'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.567153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7521e8cc9f24099b28eef50fea28b1e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.575519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23754d77014a41cbb62d7c81daaac599'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.587729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 919/1866] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/users/23754d77014a41cbb62d7c81daaac599 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.593342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7521e8cc9f24099b28eef50fea28b1e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.596139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.597661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.618617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7521e8cc9f24099b28eef50fea28b1e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.620050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7521e8cc9f24099b28eef50fea28b1e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.620876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-16056252-f506-4ae2-aa70-7136594aca6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7521e8cc9f24099b28eef50fea28b1e'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.643410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 948/1867] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/users/f7521e8cc9f24099b28eef50fea28b1e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.696450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458c490-f81a-4a76-a4ad-9f5b8d03d10d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:37.764716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:37.765368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.765659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.765944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] PATH_INFO: `/v3/projects/ddf73e410f9e43a28d4798c41124bf1f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.767485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf73e410f9e43a28d4798c41124bf1f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:37.769842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:37.822222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:37.834893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf73e410f9e43a28d4798c41124bf1f', 'request_id': 'req-38015477-824f-4d5c-ab73-4daa1a150ebb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.854728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf73e410f9e43a28d4798c41124bf1f', 'request_id': 'req-38015477-824f-4d5c-ab73-4daa1a150ebb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.858017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf73e410f9e43a28d4798c41124bf1f', 'request_id': 'req-38015477-824f-4d5c-ab73-4daa1a150ebb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.877848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] The token cache is being invalidate because project ddf73e410f9e43a28d4798c41124bf1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:37.878130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.878995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:37.884848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 920/1868] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/projects/ddf73e410f9e43a28d4798c41124bf1f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:37.886610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07d9-8ef4-420e-93f2-21f6129d4595 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:37.886910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07d9-8ef4-420e-93f2-21f6129d4595 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:37.887229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07d9-8ef4-420e-93f2-21f6129d4595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:37.906634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7c6b07d9-8ef4-420e-93f2-21f6129d4595 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:37.908004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7c6b07d9-8ef4-420e-93f2-21f6129d4595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:37.918885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:37.920428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.003569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 949/1869] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.012888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38015477-824f-4d5c-ab73-4daa1a150ebb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.013147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.013711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.013811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.022521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.023686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:38.025822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.026386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.026968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] PATH_INFO: `/v3/projects/43a403ce7beb4cf28ca1296d9579eace` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.029197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a403ce7beb4cf28ca1296d9579eace)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.032895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.037602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.040135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.049964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 950/1870] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.054188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.064845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1df2c-7099-482f-9206-79385c5db9d0 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.069835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.070769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.071118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.071533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.072524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:38.072926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.074810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.088209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 951/1871] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:15:38.092794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.101458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a403ce7beb4cf28ca1296d9579eace', 'request_id': 'req-c1a9805a-71f7-473e-a352-42276e1b201a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.107445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c719e-55d3-4643-ac5c-f56877ff9d99 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.113495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a403ce7beb4cf28ca1296d9579eace', 'request_id': 'req-c1a9805a-71f7-473e-a352-42276e1b201a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.124325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a403ce7beb4cf28ca1296d9579eace', 'request_id': 'req-c1a9805a-71f7-473e-a352-42276e1b201a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.141658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] The token cache is being invalidate because project 43a403ce7beb4cf28ca1296d9579eace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:38.142160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.142978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c1a9805a-71f7-473e-a352-42276e1b201a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.145050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 921/1872] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:37 2022] DELETE /identity/v3/projects/43a403ce7beb4cf28ca1296d9579eace => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:38.150490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.150856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.151292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.173468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.174708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.201588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.202352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.202633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.202913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.203660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.205775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.235557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:38.239885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 952/1873] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:38.239885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.239885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.250764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 922/1874] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.252073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.253785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.265673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6e4d0-f610-45bd-92e0-c5d693ea00fb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.318036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d71f6-cfdb-4493-8c25-9b5f5e0dcc2b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.322992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.324098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.324486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.324887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] PATH_INFO: `/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.326335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a094f2211e745899ebdfb4d6ae65c2a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.328746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.331840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.332661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.332922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.333211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.333964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.336356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78354963-d820-4929-aca3-62a87bcd76ee admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.364104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 923/1875] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:38.365753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15b86-0ee1-4895-9d19-c0ea724ac753 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.366050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15b86-0ee1-4895-9d19-c0ea724ac753 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.366340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc15b86-0ee1-4895-9d19-c0ea724ac753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.386872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7cc15b86-0ee1-4895-9d19-c0ea724ac753 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.388890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7cc15b86-0ee1-4895-9d19-c0ea724ac753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.401291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a094f2211e745899ebdfb4d6ae65c2a', 'request_id': 'req-7258fdec-f918-4181-ad40-b2c300469003'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.410887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a094f2211e745899ebdfb4d6ae65c2a', 'request_id': 'req-7258fdec-f918-4181-ad40-b2c300469003'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.418527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a094f2211e745899ebdfb4d6ae65c2a', 'request_id': 'req-7258fdec-f918-4181-ad40-b2c300469003'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.431184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:38.435397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] The token cache is being invalidate because project 5a094f2211e745899ebdfb4d6ae65c2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:38.435876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.436838 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:38.442070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 953/1876] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:38 2022] DELETE /identity/v3/projects/5a094f2211e745899ebdfb4d6ae65c2a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:38.442070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.442748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.482416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 924/1877] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.487543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.487981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.488335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.526757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.528153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.536964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 925/1878] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.537672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.550381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad6c4c-db84-4df7-b86c-0acdbdce4553 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.555319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7258fdec-f918-4181-ad40-b2c300469003 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.556539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.557196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.557670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.557960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.559528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:38.559837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.561760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.561982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.562621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.562915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.563256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.564005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.565689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430dc39-38c9-4503-85d0-b6038e22e699 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.576322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 954/1879] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:38.577869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.578164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.578494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.592351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 926/1880] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:38.592351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.601239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.602466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.602896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f314f1-c248-42f8-bef5-e4bfe2dab9d5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.614232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 955/1881] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.614232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.616735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.617568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.617855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.618437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.619221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.621670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.624536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c1cea-e7a0-48be-bc0c-d19fd0603cf0 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.632091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.632975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.633375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.633749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.634613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.637177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.639277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 927/1882] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:38.640385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.650702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f668ad-a500-45af-9deb-2ee20be76739 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.655918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.656777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.657068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.657357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.658098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:38.658415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.663219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.667166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 956/1883] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:38.667497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.678323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 928/1884] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:38.679394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.690307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540979ad-2047-4341-858e-6fd093cf3121 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.738644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:38.751527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4a9ea-00be-4df9-b013-23bc459c28ae admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.751713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.752352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.752641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.752939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.753640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.755609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.756690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.757600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.757888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.758174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] PATH_INFO: `/v3/projects/f693765c7b6f4f78983681e612c6a1ed/users/3635bbe92c954242aa5b34a79d3762e0/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.776643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f693765c7b6f4f78983681e612c6a1ed, user_id=3635bbe92c954242aa5b34a79d3762e0, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.779623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.784637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 929/1885] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:38.784637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.795146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af6b-d4ab-43c7-8fb0-81377fa4ad6b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.795611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 957/1886] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:38 2022] PUT /identity/v3/projects/f693765c7b6f4f78983681e612c6a1ed/users/3635bbe92c954242aa5b34a79d3762e0/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:38.796433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.852109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:38.858126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.858920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.859106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.859405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.860107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.862037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.865274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad253f-80e0-4ff1-bf5f-9db8691f7f93 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.870949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.871113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.871400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.871675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.872528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.874478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c956834-6ef3-4859-be77-8a0847b3f3a9 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.888548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 958/1887] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:38.890000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.890462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.890748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.903184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 930/1888] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/projects => generated 355 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:38.904155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.926576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] MFA Rules not processed for user `3635bbe92c954242aa5b34a79d3762e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:38.927823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:38.965767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f19731-7160-4943-b776-f826e1203253 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.970472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:38.971185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:38.971482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:38.971807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:38.972546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:38.974165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:38.984620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 931/1889] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:38.985449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.993129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 959/1890] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:38.993992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:38.994771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a99e2-f873-47d2-9b8c-0f3d0c92dc3d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:38.999852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.000516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.000804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.001086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.001769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.003719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.006228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c9a45c-ac64-4e39-884f-36dac403f9d4 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.010898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.011934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.012232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.012513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] PATH_INFO: `/v3/projects/7210054a852d484ebe70a912498be6da/users/486111d9b2f6487583d4fc37682e0f55/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.026190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 932/1891] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:38 2022] POST /identity/v3/users => generated 278 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.027301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.028922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7210054a852d484ebe70a912498be6da, user_id=486111d9b2f6487583d4fc37682e0f55, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.031876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.038953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdbafa-faa9-42e8-afbf-8f32188e9b7b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.044670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.045329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.045620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.045906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.046261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 960/1892] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:38 2022] PUT /identity/v3/projects/7210054a852d484ebe70a912498be6da/users/486111d9b2f6487583d4fc37682e0f55/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.046658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.047054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.048800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0faff2d-788d-4764-a8b6-4fb589c9a826 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.078698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 933/1893] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.080046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.080341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.080635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.106209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] MFA Rules not processed for user `486111d9b2f6487583d4fc37682e0f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:39.107557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:39.111212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6c82d-0c42-4fec-a6fe-17b4c6a4db3c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.112659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.167934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.168617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.168906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.169185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.169955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.171604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.172844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.173338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 934/1894] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:39.174373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.185872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e4c58-cb7d-4653-a36e-564c79510101 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.187049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 961/1895] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.187779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.191345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.192155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.192446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.192732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.193430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.195558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.197365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54274585-bb73-42f7-bb64-6477a7d3cb27 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.202020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.202663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.202954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.203263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.203954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.205832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.220531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 935/1896] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.220531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.230869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b6d2b4-d477-442b-aeb4-039aa4223123 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.235595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.236221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 962/1897] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.236318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.236674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.236961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.237691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.239578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.239777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.250436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 936/1898] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.251269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.260782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e9296-514f-47c5-8074-ac1af5beb802 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.305651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a9df0-df02-40be-a54b-28a5b8f0605f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.310978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.311743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.312031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.312311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.313152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.314778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.318682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.319420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.319723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.320404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.321229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.323222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.324993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 963/1899] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.325873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.335829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25b32e-69dc-444f-9261-f4ba641c3749 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.341180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.342572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.342915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.343241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] PATH_INFO: `/v3/projects/1a39c978ab4d4266a39f6f9cc8cb1c34/users/bfa5c8210ead456ca337b3e54a0fb591/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.349046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 937/1900] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.350157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.362670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a39c978ab4d4266a39f6f9cc8cb1c34, user_id=bfa5c8210ead456ca337b3e54a0fb591, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.364057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5b838-2e25-4eeb-93b4-817c76200f9b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.365527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.371410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.372505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.372926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.373267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] PATH_INFO: `/v3/projects/325e5218417547028ee4f19ff3020943/users/96fbfd8a8c614ad2b70cd17adae8284c/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.390733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 964/1901] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/1a39c978ab4d4266a39f6f9cc8cb1c34/users/bfa5c8210ead456ca337b3e54a0fb591/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.390733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.398094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=325e5218417547028ee4f19ff3020943, user_id=96fbfd8a8c614ad2b70cd17adae8284c, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.400848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371ac14-cb2f-4070-bd06-519f519802dc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.417381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 938/1902] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/325e5218417547028ee4f19ff3020943/users/96fbfd8a8c614ad2b70cd17adae8284c/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.418890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.419216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.419501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.448841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] MFA Rules not processed for user `bfa5c8210ead456ca337b3e54a0fb591`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:39.450127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:39.461662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01cb324-cd5a-47bc-a122-6f54ccfec5e7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.483430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.508708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.520779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 939/1903] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:39.520779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.521029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.521029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.521029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.521029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.521369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.522549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.528352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e1a46-19db-4a4b-af01-7c6cd7a93f02 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.532973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.536557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 965/1904] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.536683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.536683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.536683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.536683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.537256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.539186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.548421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 940/1905] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.549331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.552644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efe2eb-bfdc-41fb-a44c-ea9e65840d87 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.558353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.560304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.560846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.561226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.561354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f0ffb-3980-4487-8875-9cea65069580 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.562113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.565000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.566947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.567990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.568375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.568549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] PATH_INFO: `/v3/projects/a8f261c87fd848beaa924eff933ad89d/users/f98047c128af4033a7d9a4f2f2abd93f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.597603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f261c87fd848beaa924eff933ad89d, user_id=f98047c128af4033a7d9a4f2f2abd93f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.600548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.610074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 966/1906] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/projects => generated 355 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.610074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.618812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 941/1907] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/a8f261c87fd848beaa924eff933ad89d/users/f98047c128af4033a7d9a4f2f2abd93f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.618812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.681499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c5f5f-98c8-4edc-95ae-019e178ff333 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.684616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02004eac-c8f6-4f08-8133-27cc6ff863b1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.686912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.687922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.688330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.688614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] PATH_INFO: `/v3/projects/325e5218417547028ee4f19ff3020943/users/96fbfd8a8c614ad2b70cd17adae8284c/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.690530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.690671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] RBAC: Authorizing `identity:create_grant(project_id=325e5218417547028ee4f19ff3020943, user_id=96fbfd8a8c614ad2b70cd17adae8284c, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.691693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.691693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.691939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.692655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.693641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.694575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83423c4e-cf0c-4bbe-855c-22c0b216adf8 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.707159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 967/1908] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/325e5218417547028ee4f19ff3020943/users/96fbfd8a8c614ad2b70cd17adae8284c/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.715801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.727000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 942/1909] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:39.728338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588c11-155d-4213-af24-240c64941a45 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.728632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588c11-155d-4213-af24-240c64941a45 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.729048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f588c11-155d-4213-af24-240c64941a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.756125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5f588c11-155d-4213-af24-240c64941a45 None None] MFA Rules not processed for user `96fbfd8a8c614ad2b70cd17adae8284c`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:39.757349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5f588c11-155d-4213-af24-240c64941a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:39.781900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da86ad-3a8a-4486-8dad-407ae47617be admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.787793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.788741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.788959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.789162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.789982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.791851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.802546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 968/1910] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.803575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.814133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b180c1d-f4a7-4029-96ea-12b4222aeb6a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.818965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.819671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.819967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.820236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.820971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.822579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.833700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 969/1911] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:39.834601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.838465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 943/1912] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:39.839478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f588c11-155d-4213-af24-240c64941a45 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.845711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e68d4-7a86-4534-bd29-e7c46b38df81 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.850633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f588c11-155d-4213-af24-240c64941a45 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:39.854744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.855800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.856087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.856181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:39.856412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] PATH_INFO: `/v3/projects/a8f261c87fd848beaa924eff933ad89d/users/f98047c128af4033a7d9a4f2f2abd93f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.857115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.857406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.857688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] PATH_INFO: `/v3/projects/c629663e83864ac6975587918803bc76/users/4a03d71c08324b5e852d630c717f49a2/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.858829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f261c87fd848beaa924eff933ad89d, user_id=f98047c128af4033a7d9a4f2f2abd93f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.862655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.870449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.884045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 970/1913] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/a8f261c87fd848beaa924eff933ad89d/users/f98047c128af4033a7d9a4f2f2abd93f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.884045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:39.884315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c629663e83864ac6975587918803bc76, user_id=4a03d71c08324b5e852d630c717f49a2, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:39.884440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:39.884529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e55ff7-0564-4a39-a87d-d22fb4c391a6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:39.899745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 944/1914] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:39 2022] PUT /identity/v3/projects/c629663e83864ac6975587918803bc76/users/4a03d71c08324b5e852d630c717f49a2/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:39.901066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:39.901365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:39.903072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:39.920340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:39.941302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] MFA Rules not processed for user `f98047c128af4033a7d9a4f2f2abd93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:39.942641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:39.950976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7218319e-8deb-454b-8929-dd4132d5b62d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.007581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.008379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.008673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.008957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.032051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 945/1915] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:40.032914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.071199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.072656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.082921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 971/1916] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:40.083855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.085451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:40.097754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97883ef-2faa-4b73-9a37-782c8d403d84 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.098612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ccd78-21e1-49d1-a224-fc915fac625e service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.102523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.103308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.103593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.103894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.104037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.104587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.104695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.104918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.105192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.106231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.121649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 946/1917] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:40.121772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.123265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.124813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.132189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c3475-4eed-40f5-985f-7cb0fc1e243d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.135432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 972/1918] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:40.139010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.140065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.140352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.140636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] PATH_INFO: `/v3/projects/c629663e83864ac6975587918803bc76/users/4a03d71c08324b5e852d630c717f49a2/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.143429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c629663e83864ac6975587918803bc76, user_id=4a03d71c08324b5e852d630c717f49a2, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.147579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0cfb3-7b93-4a6e-afef-8b9719c9c2cb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.162685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 947/1919] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:40 2022] PUT /identity/v3/projects/c629663e83864ac6975587918803bc76/users/4a03d71c08324b5e852d630c717f49a2/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:40.170139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.172105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.172482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.172841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.179724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b4906e-823c-434a-8b64-933509cce559 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.202135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] MFA Rules not processed for user `4a03d71c08324b5e852d630c717f49a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:40.203402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:40.249304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.249974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.250260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.250540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.260644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:40.298550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 948/1920] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:40 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:40.301893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:40.306537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.316840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4144a12-c23c-4e9d-af1c-a62a8bdcad74 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.319620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.320571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.321062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.321204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.321489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.321769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.330262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 973/1921] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:40.331282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.332721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.342402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 949/1922] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:40.486885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:40.496530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b47ab0-f704-4c42-8e44-bd2286fffb9b service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:40.501041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:40.501763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:40.502053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:40.502335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:40.563701 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:40.565340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4e51b-e738-4cfe-90bd-ae71a7bd0fef service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:40.575222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 974/1923] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:41.839784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:41.914748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be629cd-681b-4c9d-9d8c-fe4fdfead907 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:41.920034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:41.920851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:41.921133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:41.921412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] PATH_INFO: `/v3/users/486111d9b2f6487583d4fc37682e0f55` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:41.922640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=486111d9b2f6487583d4fc37682e0f55)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:41.924715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:41.977628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '486111d9b2f6487583d4fc37682e0f55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:41.998666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '486111d9b2f6487583d4fc37682e0f55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.013668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '486111d9b2f6487583d4fc37682e0f55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.014482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '486111d9b2f6487583d4fc37682e0f55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.024435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '486111d9b2f6487583d4fc37682e0f55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.031544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 950/1924] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:41 2022] DELETE /identity/v3/users/486111d9b2f6487583d4fc37682e0f55 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:42.038084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:42.039924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:42.040684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:42.068232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:42.069988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:42.156739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 975/1925] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:42.166952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:42.168467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:42.275720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:42.290271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b096f30-33f4-4e9f-b408-ca7b61716390 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:42.300230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:42.302405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:42.303337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:42.303970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:42.327352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:42.329204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:42.341593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 951/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:42.637406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:42.639659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:42.652260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791dc25-4509-444b-ac09-8e97946827bd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:42.658387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:42.659420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:42.659770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:42.660110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] PATH_INFO: `/v3/projects/7210054a852d484ebe70a912498be6da` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:42.661829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7210054a852d484ebe70a912498be6da)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:42.663905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:42.720524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7210054a852d484ebe70a912498be6da', 'request_id': 'req-38042634-4f9a-456b-88bd-9d3d61ccab3a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.729693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7210054a852d484ebe70a912498be6da', 'request_id': 'req-38042634-4f9a-456b-88bd-9d3d61ccab3a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.737681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7210054a852d484ebe70a912498be6da', 'request_id': 'req-38042634-4f9a-456b-88bd-9d3d61ccab3a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.752631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] The token cache is being invalidate because project 7210054a852d484ebe70a912498be6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:42.753030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.755441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.757470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 976/1927] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:42 2022] DELETE /identity/v3/projects/7210054a852d484ebe70a912498be6da => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:42.761267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:42.773395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6030de1-3b99-44fa-86db-1d340a142388 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:42.853065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:42.853966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:42.854257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:42.854538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] PATH_INFO: `/v3/users/f98047c128af4033a7d9a4f2f2abd93f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:42.855760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98047c128af4033a7d9a4f2f2abd93f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:42.858040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:42.908132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98047c128af4033a7d9a4f2f2abd93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.916004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98047c128af4033a7d9a4f2f2abd93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.929790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98047c128af4033a7d9a4f2f2abd93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.930579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98047c128af4033a7d9a4f2f2abd93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.940113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d4a640e6-a649-4db8-afc7-c2de637988ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98047c128af4033a7d9a4f2f2abd93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:42.946550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 952/1928] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:42 2022] DELETE /identity/v3/users/f98047c128af4033a7d9a4f2f2abd93f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:43.345087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.346566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:43.355026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:43.361111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38042634-4f9a-456b-88bd-9d3d61ccab3a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.427600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.428604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.428897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.429179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] PATH_INFO: `/v3/projects/a8f261c87fd848beaa924eff933ad89d` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.430378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f261c87fd848beaa924eff933ad89d)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.432326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.480881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f261c87fd848beaa924eff933ad89d', 'request_id': 'req-c5096041-5be7-4882-8824-6b05ac3dc758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:43.489535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f261c87fd848beaa924eff933ad89d', 'request_id': 'req-c5096041-5be7-4882-8824-6b05ac3dc758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:43.495412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f261c87fd848beaa924eff933ad89d', 'request_id': 'req-c5096041-5be7-4882-8824-6b05ac3dc758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:43.507054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] The token cache is being invalidate because project a8f261c87fd848beaa924eff933ad89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:43.507360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:43.508016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5096041-5be7-4882-8824-6b05ac3dc758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:43.509711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 977/1929] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:43 2022] DELETE /identity/v3/projects/a8f261c87fd848beaa924eff933ad89d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:43.515227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.515523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.515806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.535662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:43.536906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:43.599524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 953/1930] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:43.607621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.607944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.608226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.626401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:43.627636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:43.635469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 978/1931] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:43.640252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.654677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97192a6e-397b-4609-a05d-9a8d433c3912 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.661667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.662689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.663087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.663560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.664680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:43.665145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.667753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.677983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:43.685595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 954/1932] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:43.690241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.699878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a14fa-2bc5-4536-b2de-42513012079d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.705311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.705966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.706244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.706519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.707232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.709155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.738040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 979/1933] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:43 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:43.742040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.752583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25787f10-1158-4003-afa3-974ab8fafe7d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.820126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.820817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.821100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.821384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.822158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.824130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.853392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 955/1934] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:43 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:43.857994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.873093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed9415-3a99-43fa-9292-f70aa6cc7d3a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.879596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.881959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.882913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.883621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.885089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.887025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.898516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 980/1935] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:43.902615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.914025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d3a75-67e8-4fce-bb91-e34087857902 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.918711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.919494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.919917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.920202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.920965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.922628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79ab8f-5ce1-4b14-a25e-74c0f003240a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.932954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 956/1936] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:43.937734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:43.947952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40553b-f6c4-4cd8-b521-add8a47571f9 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:43.954051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:43.954982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:43.955319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:43.955592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] PATH_INFO: `/v3/projects/3b2b95a591544ae5abfe66a9f14680f0/users/e3bc4ef5a75943b382f78fa005611d5b/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:43.972051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2b95a591544ae5abfe66a9f14680f0, user_id=e3bc4ef5a75943b382f78fa005611d5b, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:43.974904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:43.989686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 981/1937] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:43 2022] PUT /identity/v3/projects/3b2b95a591544ae5abfe66a9f14680f0/users/e3bc4ef5a75943b382f78fa005611d5b/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:44.001311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.001533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.001821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.029101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] MFA Rules not processed for user `e3bc4ef5a75943b382f78fa005611d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:44.030579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:44.100013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 957/1938] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:44.104705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.136995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:44.172903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592afe48-0e12-4f5e-a779-fc04d674c6cd admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.178397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.179820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.180308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.180775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.182112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.185862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.216465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 982/1939] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:44 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:44.221296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.233015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ec035-4dc4-4516-8909-5b8175f16130 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.298515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.299145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.299450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.299731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.300428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.302404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.312009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:44.327387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 958/1940] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:44 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:44.332712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.344058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768acb63-46b6-452f-92d4-884110ee64e8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.349994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.350681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.350969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.351320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.352103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.353864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.365649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 983/1941] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:44.370046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.381369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e4545-bd08-461e-9f88-cfdd3240a77e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.386544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.387490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.387781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.388062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] PATH_INFO: `/v3/projects/5b273a203325401f8863d61078f4e1af/users/ede7f30372d94a73995229ae276a45cd/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.409304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b273a203325401f8863d61078f4e1af, user_id=ede7f30372d94a73995229ae276a45cd, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.412997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.432860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 959/1942] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:44 2022] PUT /identity/v3/projects/5b273a203325401f8863d61078f4e1af/users/ede7f30372d94a73995229ae276a45cd/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:44.436795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.447435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a3f22-ebbd-4d07-b46d-84015540c13b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.504988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.505664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.505952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.506240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.506986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.508636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d289a7-0546-4bce-a676-4cd5ea722109 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.518787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 984/1943] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:44.523003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.534307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f6a51-4aca-4f09-b6ed-3d34e546c61b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.539908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.541261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.541765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.542222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] PATH_INFO: `/v3/projects/5b273a203325401f8863d61078f4e1af/users/ede7f30372d94a73995229ae276a45cd/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.544540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b273a203325401f8863d61078f4e1af, user_id=ede7f30372d94a73995229ae276a45cd, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.547391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.561138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 960/1944] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:44 2022] PUT /identity/v3/projects/5b273a203325401f8863d61078f4e1af/users/ede7f30372d94a73995229ae276a45cd/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:44.572537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.572833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.573109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.602480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] MFA Rules not processed for user `ede7f30372d94a73995229ae276a45cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:44.603738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:44.673355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 985/1945] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:44.684719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.686632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:44.745750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494255b-3a37-441f-9bbe-612b111941f7 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.749507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.750173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.750459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.750739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.760557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.762065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfa874-add9-4415-a61c-b73681eee6f9 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:44.773677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 961/1946] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:44.854669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:44.885244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:44.940057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32d7d8-c744-4ab2-8c65-c90d9f0a2a3a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:44.946460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:44.947475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:44.947826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:44.948156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] PATH_INFO: `/v3/users/e3bc4ef5a75943b382f78fa005611d5b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:44.949432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bc4ef5a75943b382f78fa005611d5b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:44.951781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:45.009040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bc4ef5a75943b382f78fa005611d5b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.023684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bc4ef5a75943b382f78fa005611d5b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.032879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bc4ef5a75943b382f78fa005611d5b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.033615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bc4ef5a75943b382f78fa005611d5b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.034842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bc4ef5a75943b382f78fa005611d5b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.050301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 986/1947] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:44 2022] DELETE /identity/v3/users/e3bc4ef5a75943b382f78fa005611d5b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:45.055048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:45.055585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:45.055814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:45.073304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:45.074503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:45.133213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 962/1948] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:45.144509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:45.145962 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:45.205520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20be2b-6f54-434f-9caf-0420b6040e18 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:45.210276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:45.210944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:45.211261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:45.211541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:45.221407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:45.222899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:45.233362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 987/1949] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:45.505036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:45.506679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:45.541111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d6e02-69c5-4b3d-9c68-b19b531bab6e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:45.547423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:45.548373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:45.548599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:45.548879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] PATH_INFO: `/v3/projects/3b2b95a591544ae5abfe66a9f14680f0` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:45.550092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2b95a591544ae5abfe66a9f14680f0)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:45.552087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:45.609940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2b95a591544ae5abfe66a9f14680f0', 'request_id': 'req-ea49a44b-4685-49e3-a0c6-767a202aea55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.618392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2b95a591544ae5abfe66a9f14680f0', 'request_id': 'req-ea49a44b-4685-49e3-a0c6-767a202aea55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.619851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:45.625688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2b95a591544ae5abfe66a9f14680f0', 'request_id': 'req-ea49a44b-4685-49e3-a0c6-767a202aea55'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.638278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] The token cache is being invalidate because project 3b2b95a591544ae5abfe66a9f14680f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:45.638607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.639373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.641285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 963/1950] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:45 2022] DELETE /identity/v3/projects/3b2b95a591544ae5abfe66a9f14680f0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:45.645111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:45.655635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e40e1-6c41-4d1f-a467-73ea203971f8 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:45.702265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:45.716477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:45.717551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:45.717854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:45.718116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] PATH_INFO: `/v3/users/ede7f30372d94a73995229ae276a45cd` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:45.719323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede7f30372d94a73995229ae276a45cd)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:45.721262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:45.771219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede7f30372d94a73995229ae276a45cd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.786074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede7f30372d94a73995229ae276a45cd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.792982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede7f30372d94a73995229ae276a45cd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.793741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede7f30372d94a73995229ae276a45cd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.794456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6a87ffa3-9695-4196-ae4d-dd285640653a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede7f30372d94a73995229ae276a45cd'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:45.808422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 988/1951] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:45 2022] DELETE /identity/v3/users/ede7f30372d94a73995229ae276a45cd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:46.082620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.084417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:46.099829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49a44b-4685-49e3-a0c6-767a202aea55 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.156733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.157535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.157796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.158076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] PATH_INFO: `/v3/projects/5b273a203325401f8863d61078f4e1af` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.159330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b273a203325401f8863d61078f4e1af)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.161665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.207560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b273a203325401f8863d61078f4e1af', 'request_id': 'req-a50871d3-555d-4391-93c8-757bbc54937b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:46.217306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b273a203325401f8863d61078f4e1af', 'request_id': 'req-a50871d3-555d-4391-93c8-757bbc54937b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:46.224862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b273a203325401f8863d61078f4e1af', 'request_id': 'req-a50871d3-555d-4391-93c8-757bbc54937b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:46.237539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] The token cache is being invalidate because project 5b273a203325401f8863d61078f4e1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:46.237806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:46.238575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a50871d3-555d-4391-93c8-757bbc54937b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:46.240260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 964/1952] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:46 2022] DELETE /identity/v3/projects/5b273a203325401f8863d61078f4e1af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:46.245114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3146-8598-4e22-958a-becd67860637 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.245519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3146-8598-4e22-958a-becd67860637 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.245801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db3146-8598-4e22-958a-becd67860637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.272009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-73db3146-8598-4e22-958a-becd67860637 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:46.274053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-73db3146-8598-4e22-958a-becd67860637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:46.336238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 989/1953] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:46.345184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.345481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.345766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.364881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:46.366101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:46.374319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 965/1954] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:46.378799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3146-8598-4e22-958a-becd67860637 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.389562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db3146-8598-4e22-958a-becd67860637 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.394804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.395490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.395864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.396145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.396881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:46.397197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.398813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.411436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 990/1955] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:46.415705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.425347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2d9b1-53e8-4682-a9f4-9dbc93734e1c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.431151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.431823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.432131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.432405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.433110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.435028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.463623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 966/1956] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:46.467987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.478493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31269b25-a78d-495e-a75e-347f8d77eb22 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.530170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:46.541185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.542020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.542435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.542763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.543676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.546058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.567479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 991/1957] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:46.571878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.586600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af109c0a-4984-49fe-9c00-877fe4c22fcd admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.593186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.593870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.594155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.594433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.595225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.596851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.609579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 967/1958] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:46.618287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.628895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9334-4122-4a2b-a047-4842b497cf5a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.634086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.634755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.635066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.635370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.636132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.637826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3249fb-9695-4ba2-adc0-c0f206fe2607 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.647949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 992/1959] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:46.652075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.663509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bddca-eb1d-43e3-9de0-2ef4fb38052e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.668382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.669268 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.670008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.670293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] PATH_INFO: `/v3/projects/61c7f438acc848b6921830d196489b34/users/31bc5b7429d74825b0a470c87abe451f/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.686107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61c7f438acc848b6921830d196489b34, user_id=31bc5b7429d74825b0a470c87abe451f, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.688831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.704502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 968/1960] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:46 2022] PUT /identity/v3/projects/61c7f438acc848b6921830d196489b34/users/31bc5b7429d74825b0a470c87abe451f/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:46.715026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.715447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.715755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.744508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] MFA Rules not processed for user `31bc5b7429d74825b0a470c87abe451f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:46.745874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:46.816925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 993/1961] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:46.821491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.834411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:46.889722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff6b4a-c145-4adb-b741-d9a147d54f90 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:46.894797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:46.895498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:46.895774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:46.896057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:46.896759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:46.898653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:46.924862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 969/1962] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:46.929099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:46.940437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d538340-c664-4754-b0a7-f7e87b7de759 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.007294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.007986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.008301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.008543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.009261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.011214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.036946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 994/1963] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:46 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:47.041241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.059444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b021f9d-b83a-42e4-96fc-8588a38b24c8 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.065825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.066694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.066821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.067084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.067860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.069628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.082044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 970/1964] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:47.086328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.097218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487eb0b-148a-4331-a378-843864935c2c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.102609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.103664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.104020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.104397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] PATH_INFO: `/v3/projects/7163fd5aaacc407689e99000e0c2f9bd/users/bf515d7088734399853d991e4aa8d296/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.126007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7163fd5aaacc407689e99000e0c2f9bd, user_id=bf515d7088734399853d991e4aa8d296, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.132948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0c9f4-1cf2-4d6e-b7f5-9097f61283a7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.149879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 995/1965] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:47 2022] PUT /identity/v3/projects/7163fd5aaacc407689e99000e0c2f9bd/users/bf515d7088734399853d991e4aa8d296/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:47.162186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.173417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea37e2ad-9bd1-4c19-91a7-e6c0e07b63ab admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.230367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.231267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.231552 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.231869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.232621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.234309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7779543-1a44-4396-9a73-4034fd843a38 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.240451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572a210-e2e4-45d1-a7cd-4d29567e6b79 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.240966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572a210-e2e4-45d1-a7cd-4d29567e6b79 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.241488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572a210-e2e4-45d1-a7cd-4d29567e6b79 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.244093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 996/1966] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:15:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:15:47.245761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 971/1967] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:47.250255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572a210-e2e4-45d1-a7cd-4d29567e6b79 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.253330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.253330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.253330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.268153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572a210-e2e4-45d1-a7cd-4d29567e6b79 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.273257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:47.273594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.274385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:47.274622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.274989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.275347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] PATH_INFO: `/v3/projects/7163fd5aaacc407689e99000e0c2f9bd/users/bf515d7088734399853d991e4aa8d296/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.279334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7163fd5aaacc407689e99000e0c2f9bd, user_id=bf515d7088734399853d991e4aa8d296, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.281626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a07197-e0c8-402a-8693-d27ef806358e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.295982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 997/1968] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:47 2022] PUT /identity/v3/projects/7163fd5aaacc407689e99000e0c2f9bd/users/bf515d7088734399853d991e4aa8d296/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:47.306845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.307163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.307861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.348060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:47.352903 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] MFA Rules not processed for user `bf515d7088734399853d991e4aa8d296`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:47.358752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:47.381352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 972/1969] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:15:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:47.405657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.408133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:47.425679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65ff76-74ee-458d-bd0d-cf0eacc83777 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.439434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:47.446744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 998/1970] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:47.457342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.459160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:47.517232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.518131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.518487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.518768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.539557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.539981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159fbb7-f97c-42bf-b8f8-d8ef2055a8ff None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.543206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.545530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.546568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.547922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.548355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.561390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 973/1971] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:47.562092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.563760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.583532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 999/1972] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:47.761334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:47.772738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98575a3-8e5e-4e29-8dbe-f5b459204a24 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:47.779175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:47.779854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:47.780144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:47.780427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:47.847168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:47.848723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7b9b6-a396-42ec-8d47-3eab301fa21a service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:47.862851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 974/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:48.263568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:48.266325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:48.367315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269cf001-1319-4f45-b9ec-8a70bfdd0e22 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:48.374826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:48.375773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:48.376060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:48.376368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] PATH_INFO: `/v3/users/3bf5a65cfddf4fe19ed8ec7073b8c2c9` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:48.377740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf5a65cfddf4fe19ed8ec7073b8c2c9)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:48.379960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:48.446550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:48.449658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf5a65cfddf4fe19ed8ec7073b8c2c9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:48.478285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf5a65cfddf4fe19ed8ec7073b8c2c9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:48.486716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf5a65cfddf4fe19ed8ec7073b8c2c9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:48.487529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf5a65cfddf4fe19ed8ec7073b8c2c9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:48.488219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf5a65cfddf4fe19ed8ec7073b8c2c9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:48.507201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1000/1974] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:48 2022] DELETE /identity/v3/users/3bf5a65cfddf4fe19ed8ec7073b8c2c9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:48.511745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:48.512333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:48.512422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:48.531416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:48.533401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:48.636183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 975/1975] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:48.646215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:48.647746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:48.712040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64cc29-19f9-468d-8467-8f57f6c6737a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:48.716322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:48.717139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:48.717492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:48.717799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:48.728460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:48.729953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d8d49-5cd8-4c13-867f-8ac2ea25d6c5 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:48.740918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1001/1976] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:49.036718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.038248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:49.051282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1013e02-a2c4-4d4e-9003-e7bc1ebce719 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.056151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.057065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.057354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.057656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] PATH_INFO: `/v3/projects/58cb41b3cc0b44339f82fbb494180985` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.059027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58cb41b3cc0b44339f82fbb494180985)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.061109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.099462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:49.113219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58cb41b3cc0b44339f82fbb494180985', 'request_id': 'req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:49.121756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58cb41b3cc0b44339f82fbb494180985', 'request_id': 'req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:49.128241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58cb41b3cc0b44339f82fbb494180985', 'request_id': 'req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:49.140136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] The token cache is being invalidate because project 58cb41b3cc0b44339f82fbb494180985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:49.140426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:49.141669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3cd238a0-bf97-4ef9-8922-89d8a1b7be5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:49.143342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 976/1977] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:49 2022] DELETE /identity/v3/projects/58cb41b3cc0b44339f82fbb494180985 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:49.148697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.148993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.149280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.167914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:49.169205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:49.233121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1002/1978] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:49.241750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.242048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.242361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.262675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:49.264325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:49.272989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 977/1979] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:49.277435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.288968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b9301-fe41-4435-82ea-a800bff50ec9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.294541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.295638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.296154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.296667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.297430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:49.297744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.299415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.314517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1003/1980] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:49.318921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.329003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874bbbf-08a8-43f6-b9e6-613f51cef141 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.333975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.334637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.334922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.335240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.335953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.337926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.369517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 978/1981] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:49 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:49.376448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.386787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d274e8-5307-472a-8440-54782975db32 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.462447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:49.474112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.474809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.475116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.475397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.476485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.479208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.515991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1004/1982] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:49 2022] POST /identity/v3/users => generated 281 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:49.520153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.534235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece34e5a-270d-4377-a1c6-5f8cdf2dd754 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.540500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.541875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.542553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.543884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.545577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.547571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.560504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 979/1983] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:49.564156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.577543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a98488-73d9-49a7-bb07-9891a23770b7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.596100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.597022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.597435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.597782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.598707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.600566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99aec6-24f9-4d36-b18d-2556424cc9c8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.619587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1005/1984] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:49.621354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.631969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d12d7-68c1-4708-9300-b3777eb00a30 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.637863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.639163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.639461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.640198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] PATH_INFO: `/v3/projects/9cd80d08880e4be6a9ff420589f8c02e/users/d2d4007c3d884a19a3c70c4b91b401e6/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.658257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd80d08880e4be6a9ff420589f8c02e, user_id=d2d4007c3d884a19a3c70c4b91b401e6, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.661607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.679980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 980/1985] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:49 2022] PUT /identity/v3/projects/9cd80d08880e4be6a9ff420589f8c02e/users/d2d4007c3d884a19a3c70c4b91b401e6/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:49.693660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.693920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.694257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.727528 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] MFA Rules not processed for user `d2d4007c3d884a19a3c70c4b91b401e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:49.728787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:49.808346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1006/1986] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:49.847265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:49.848975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:49.929179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca01aa-3322-42b8-ae9e-47357d40ee05 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:49.934070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:49.934908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:49.935512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:49.936389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:49.946248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:49.947996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7af39e-04f3-4880-a3b2-05bd44b8555d service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:49.958853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 981/1987] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:49.974946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:50.053146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b68b6-3f30-4803-9eea-570f183ae041 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:50.060782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:50.061688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:50.061965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:50.062242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] PATH_INFO: `/v3/users/d2d4007c3d884a19a3c70c4b91b401e6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:50.063558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2d4007c3d884a19a3c70c4b91b401e6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:50.065533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:50.126692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2d4007c3d884a19a3c70c4b91b401e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.141152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:50.144647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2d4007c3d884a19a3c70c4b91b401e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.157275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2d4007c3d884a19a3c70c4b91b401e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.157889 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2d4007c3d884a19a3c70c4b91b401e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.158622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2d4007c3d884a19a3c70c4b91b401e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.182249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1007/1988] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:49 2022] DELETE /identity/v3/users/d2d4007c3d884a19a3c70c4b91b401e6 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:50.184461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:50.184854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:50.185263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:50.204321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:50.205541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:50.286519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:50.312063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 982/1989] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:50.317816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:50.319439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:50.407428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecf318-218b-43c7-92ab-689155b4c03c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:50.411767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:50.412523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:50.413037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:50.413525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:50.431592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:50.433151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:50.447981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1008/1990] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:50.790366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:50.791990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:50.809120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96389c-7098-4dc1-95e6-1464bf8ef2a8 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:50.822106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:50.823334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:50.823933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:50.824560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] PATH_INFO: `/v3/users/31bc5b7429d74825b0a470c87abe451f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:50.826390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bc5b7429d74825b0a470c87abe451f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:50.829772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:50.878723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:50.880594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:50.905221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bc5b7429d74825b0a470c87abe451f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.912977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b789b0-b1d2-46d8-8cb4-d428012a4ad0 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:50.929972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:50.930879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:50.931264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:50.931546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] PATH_INFO: `/v3/projects/9cd80d08880e4be6a9ff420589f8c02e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:50.932949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bc5b7429d74825b0a470c87abe451f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.934960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd80d08880e4be6a9ff420589f8c02e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:50.937294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:50.953636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bc5b7429d74825b0a470c87abe451f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.955048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bc5b7429d74825b0a470c87abe451f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.970009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bc5b7429d74825b0a470c87abe451f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:50.977583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 983/1991] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:50 2022] DELETE /identity/v3/users/31bc5b7429d74825b0a470c87abe451f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:50.978814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:50.980470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.018455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd80d08880e4be6a9ff420589f8c02e', 'request_id': 'req-501fdb95-2c22-4536-8d7e-9f5c30651f2b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.049023 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd80d08880e4be6a9ff420589f8c02e', 'request_id': 'req-501fdb95-2c22-4536-8d7e-9f5c30651f2b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.058031 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db12589-ce94-416b-809b-89694ca16adf admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.062456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd80d08880e4be6a9ff420589f8c02e', 'request_id': 'req-501fdb95-2c22-4536-8d7e-9f5c30651f2b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.080041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] The token cache is being invalidate because project 9cd80d08880e4be6a9ff420589f8c02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:51.080505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.081413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-501fdb95-2c22-4536-8d7e-9f5c30651f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.084187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1009/1992] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:50 2022] DELETE /identity/v3/projects/9cd80d08880e4be6a9ff420589f8c02e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:51.086861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771639e-9d5b-4af2-b4dc-69de79934f53 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.095569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771639e-9d5b-4af2-b4dc-69de79934f53 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.096102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771639e-9d5b-4af2-b4dc-69de79934f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.117403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4771639e-9d5b-4af2-b4dc-69de79934f53 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:51.118909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4771639e-9d5b-4af2-b4dc-69de79934f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.137204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.138617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.139338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.139788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] PATH_INFO: `/v3/users/5e09e89dd40146debeb07218ec466fcd` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.141330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e09e89dd40146debeb07218ec466fcd)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.144049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.177224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:51.214680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1010/1993] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:51.216206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.216530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.216815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.235315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:51.238609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e09e89dd40146debeb07218ec466fcd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.251547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e09e89dd40146debeb07218ec466fcd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.254291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:51.257087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.277094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e09e89dd40146debeb07218ec466fcd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.277094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e09e89dd40146debeb07218ec466fcd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.290020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1011/1994] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:51.291125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.293018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.312351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9245168f-69de-4b25-9e50-bd90024e98d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e09e89dd40146debeb07218ec466fcd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:51.322879 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 984/1995] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:50 2022] DELETE /identity/v3/users/5e09e89dd40146debeb07218ec466fcd => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:51.327768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798597bf-4c80-49a2-8e3a-6339f01aec1a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.328179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798597bf-4c80-49a2-8e3a-6339f01aec1a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.330067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798597bf-4c80-49a2-8e3a-6339f01aec1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.361669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329a9df-6c60-47e0-bc95-d8cecb9d2759 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.370866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-798597bf-4c80-49a2-8e3a-6339f01aec1a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:51.372634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-798597bf-4c80-49a2-8e3a-6339f01aec1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.446034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.447195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.447195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.447442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.449563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.496229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 985/1996] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:51.497936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.498074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.498606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.546300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:51.548987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.564890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 986/1997] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:51.565936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.590147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341d84-b379-4e18-b701-894e5d054c46 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.596064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.597828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.606097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.606838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.607147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.607428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.608163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:51.608468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.610082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.624898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1012/1998] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:51.625605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.628612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:51.637409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 987/1999] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:51.642419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.648067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4788517c-9352-41b2-99a7-e568dbd375d6 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.655121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bff4-8765-40b7-84b4-b31ac83792df admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.657801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.658477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.658758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.659085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.670005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.670783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.671064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.671378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.672086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.674057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.681845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.683117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.705699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1013/2000] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:51.724181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 988/2001] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:51.728662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.740605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02978216-88cc-44ae-aa44-c1d6d04019a5 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.803283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:51.819951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.820667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.827334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.827721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.828507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.830503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.860334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1014/2002] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:51 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:15:51.864267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.875992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ad1f9-752d-47ec-a9e6-b41db653c599 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.882839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.884163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.884608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.885023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.885982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.887888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.904511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 989/2003] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:51.909166 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:51.925854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066b79f-8136-4d9a-9ab4-80ddbb7b590d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:51.936647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:51.941224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:51.941515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:51.941807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:51.942574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:51.947449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87738438-8c98-4c08-99b6-c3771893e65c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:51.985228 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1015/2004] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:51.987061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.001311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246348-01a0-4151-aac0-ee47a6802f84 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.016156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.017204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.017492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.017770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] PATH_INFO: `/v3/projects/3f9bce41dfc54cf6b5ef9cae58d8b753/users/85302e62962f4d9ba8f95ea2b4c6b3ef/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.064039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9bce41dfc54cf6b5ef9cae58d8b753, user_id=85302e62962f4d9ba8f95ea2b4c6b3ef, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.066774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:52.101646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 990/2005] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:51 2022] PUT /identity/v3/projects/3f9bce41dfc54cf6b5ef9cae58d8b753/users/85302e62962f4d9ba8f95ea2b4c6b3ef/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:52.114950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.115540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.115955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.163983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] MFA Rules not processed for user `85302e62962f4d9ba8f95ea2b4c6b3ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:52.165414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.287061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1016/2006] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:52.301392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.302988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.330653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:52.374979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b181c-8957-443f-9546-f3f284f00123 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.381774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.382506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.383256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.383544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.384181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.385940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.399664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.401633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:52.416248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 991/2007] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:52.493135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237669e-743e-49bb-9cf5-7596f860b496 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.513506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.516009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.516330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.516586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] PATH_INFO: `/v3/users/3635bbe92c954242aa5b34a79d3762e0` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.519697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3635bbe92c954242aa5b34a79d3762e0)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.523229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:52.524557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.525767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.550625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a405cd4-987d-4f75-9e6f-f52ce718c3bc service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.562720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.563779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.563931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.564209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] PATH_INFO: `/v3/projects/9953e3ce880a4f1fbc900eb606d315ee` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.571232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9953e3ce880a4f1fbc900eb606d315ee)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.573261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:52.663346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3635bbe92c954242aa5b34a79d3762e0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.669530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9953e3ce880a4f1fbc900eb606d315ee', 'request_id': 'req-540358ad-3fb7-42ac-b9a4-c34026a90029'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.686377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9953e3ce880a4f1fbc900eb606d315ee', 'request_id': 'req-540358ad-3fb7-42ac-b9a4-c34026a90029'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.690198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:52.698278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9953e3ce880a4f1fbc900eb606d315ee', 'request_id': 'req-540358ad-3fb7-42ac-b9a4-c34026a90029'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.698584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3635bbe92c954242aa5b34a79d3762e0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.713699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] The token cache is being invalidate because project 9953e3ce880a4f1fbc900eb606d315ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:52.714003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.715157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.715985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3635bbe92c954242aa5b34a79d3762e0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.716685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3635bbe92c954242aa5b34a79d3762e0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.720239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3635bbe92c954242aa5b34a79d3762e0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:52.720416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 992/2008] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:52 2022] DELETE /identity/v3/projects/9953e3ce880a4f1fbc900eb606d315ee => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:52.720416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.720416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.743726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1017/2009] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:52 2022] DELETE /identity/v3/users/3635bbe92c954242aa5b34a79d3762e0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:52.744754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:52.748823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:52.822374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540358ad-3fb7-42ac-b9a4-c34026a90029 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.890808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b47ff4-ee38-4546-a582-d69afdc11820 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:52.900277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.904420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.904849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.905403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] PATH_INFO: `/v3/users/7c011733f3f44b06a2644d8d94f67d98` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.908115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c011733f3f44b06a2644d8d94f67d98)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.912882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:52.929852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:52.955507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:52.956536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:52.956952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:52.957242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] PATH_INFO: `/v3/projects/61c7f438acc848b6921830d196489b34` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:52.958540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61c7f438acc848b6921830d196489b34)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:52.960966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:53.048159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c011733f3f44b06a2644d8d94f67d98'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.054195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61c7f438acc848b6921830d196489b34', 'request_id': 'req-edac6564-9d20-47bb-9897-945a12ce79c2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.066675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61c7f438acc848b6921830d196489b34', 'request_id': 'req-edac6564-9d20-47bb-9897-945a12ce79c2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.069645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c011733f3f44b06a2644d8d94f67d98'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.075815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61c7f438acc848b6921830d196489b34', 'request_id': 'req-edac6564-9d20-47bb-9897-945a12ce79c2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.079163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c011733f3f44b06a2644d8d94f67d98'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.080655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c011733f3f44b06a2644d8d94f67d98'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.081879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c011733f3f44b06a2644d8d94f67d98'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.093863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] The token cache is being invalidate because project 61c7f438acc848b6921830d196489b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:53.094187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.095019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-edac6564-9d20-47bb-9897-945a12ce79c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.096862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 993/2010] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:52 2022] DELETE /identity/v3/projects/61c7f438acc848b6921830d196489b34 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:53.101209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:53.101928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:53.103081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:53.110001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1018/2011] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:52 2022] DELETE /identity/v3/users/7c011733f3f44b06a2644d8d94f67d98 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:53.114064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:53.115804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:53.162329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:53.164001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:53.339270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 994/2012] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:53.346125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:53.351163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd288045-8444-4e52-bbf7-a9b0258be80d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:53.355131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:53.357327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:53.390548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:53.392518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:53.392944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:53.393465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] PATH_INFO: `/v3/users/bf515d7088734399853d991e4aa8d296` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:53.395358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf515d7088734399853d991e4aa8d296)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:53.398395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:53.517054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf515d7088734399853d991e4aa8d296'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.517513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0156fd24-f400-439c-9f6f-55190315eddf None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:53.529456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:53.530402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:53.530839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:53.532485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:53.542974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf515d7088734399853d991e4aa8d296'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.559468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:53.559989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf515d7088734399853d991e4aa8d296'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.561131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf515d7088734399853d991e4aa8d296'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.562366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:53.571344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e9473e9e-7402-48fd-9119-4fd4b7c7069b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf515d7088734399853d991e4aa8d296'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:53.581498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 995/2013] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:53.599193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1019/2014] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:53 2022] DELETE /identity/v3/users/bf515d7088734399853d991e4aa8d296 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:53.965889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:53.968274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.007999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e87d3e-4b35-435f-a9b3-d5315858f835 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:54.088015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:54.130084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:54.131403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.133039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.133039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] PATH_INFO: `/v3/projects/9c2442c39eca424eb256fb301065b178` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.133699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c2442c39eca424eb256fb301065b178)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:54.136411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:54.211592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c2442c39eca424eb256fb301065b178', 'request_id': 'req-eff8447a-7143-41b9-aa56-c26d3863010a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.223178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c2442c39eca424eb256fb301065b178', 'request_id': 'req-eff8447a-7143-41b9-aa56-c26d3863010a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.235741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c2442c39eca424eb256fb301065b178', 'request_id': 'req-eff8447a-7143-41b9-aa56-c26d3863010a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.269205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] The token cache is being invalidate because project 9c2442c39eca424eb256fb301065b178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:54.269205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.270443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.272204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 996/2015] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:53 2022] DELETE /identity/v3/projects/9c2442c39eca424eb256fb301065b178 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:54.277588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.278191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.278552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.320603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:54.320946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:54.322118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.324237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.430688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1020/2016] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:54.431601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:54.433018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.447718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41495e-b61e-48c0-a547-2bf925472467 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:54.454828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:54.456242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.456685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8447a-7143-41b9-aa56-c26d3863010a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:54.458615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.459117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] PATH_INFO: `/v3/projects/f693765c7b6f4f78983681e612c6a1ed` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.460679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=f693765c7b6f4f78983681e612c6a1ed)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:54.462645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:54.463366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:54.463672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.463964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.464233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] PATH_INFO: `/v3/projects/7163fd5aaacc407689e99000e0c2f9bd` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.465731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7163fd5aaacc407689e99000e0c2f9bd)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:54.468785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:54.557466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f693765c7b6f4f78983681e612c6a1ed', 'request_id': 'req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.562987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7163fd5aaacc407689e99000e0c2f9bd', 'request_id': 'req-e0803b94-b1ec-4b62-bf61-1b9409407819'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.575770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f693765c7b6f4f78983681e612c6a1ed', 'request_id': 'req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.582447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7163fd5aaacc407689e99000e0c2f9bd', 'request_id': 'req-e0803b94-b1ec-4b62-bf61-1b9409407819'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.593419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f693765c7b6f4f78983681e612c6a1ed', 'request_id': 'req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.605932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:54.607621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:54.610794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7163fd5aaacc407689e99000e0c2f9bd', 'request_id': 'req-e0803b94-b1ec-4b62-bf61-1b9409407819'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.613654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] The token cache is being invalidate because project f693765c7b6f4f78983681e612c6a1ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:54.614278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.615895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-92a28a6b-11ed-4b9a-9717-9ff9d7d964ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.630085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1021/2017] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:54 2022] DELETE /identity/v3/projects/f693765c7b6f4f78983681e612c6a1ed => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:54.631122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] The token cache is being invalidate because project 7163fd5aaacc407689e99000e0c2f9bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:54.631599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.631958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355806a4-c961-46a5-ab6e-713b8d53d5d6 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.632348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355806a4-c961-46a5-ab6e-713b8d53d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.632730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355806a4-c961-46a5-ab6e-713b8d53d5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.632913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.634902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 997/2018] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:54 2022] DELETE /identity/v3/projects/7163fd5aaacc407689e99000e0c2f9bd => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:54.635841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:54.639276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.678262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-355806a4-c961-46a5-ab6e-713b8d53d5d6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:54.680843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-355806a4-c961-46a5-ab6e-713b8d53d5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.734767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0803b94-b1ec-4b62-bf61-1b9409407819 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:54.739919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:54.740818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.741098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.741375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] PATH_INFO: `/v3/users/96fbfd8a8c614ad2b70cd17adae8284c` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.743028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=96fbfd8a8c614ad2b70cd17adae8284c)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:54.746542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:54.791400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1022/2019] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:54.793018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.793450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.793812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.814905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96fbfd8a8c614ad2b70cd17adae8284c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.819335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:54.820702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.826876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96fbfd8a8c614ad2b70cd17adae8284c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.839609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1023/2020] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:54.839609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:54.844472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c7702-c33a-4622-b28c-33d261f8083d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:54.849071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:54.850101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.850101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.850318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.851297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:54.851620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:54.853449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346bb2-14a0-424c-8964-1a2880a153a7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:54.870935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96fbfd8a8c614ad2b70cd17adae8284c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.872144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96fbfd8a8c614ad2b70cd17adae8284c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.874158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1024/2021] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:54.876279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:54.876541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:54.877086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:54.894108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96fbfd8a8c614ad2b70cd17adae8284c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:54.900363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:54.901773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:54.903812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 998/2022] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:54 2022] DELETE /identity/v3/users/96fbfd8a8c614ad2b70cd17adae8284c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:54.904326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:54.917591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.001378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1025/2023] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:55.005881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.016710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d5d7-bf40-4399-b1ad-4fe1b4ade96c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.025957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.026596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.027434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.027734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.028487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:55.028870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.030566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.031636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51c23-84f8-4884-96d7-dbfb54f21a4d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.044711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.045646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1026/2024] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:55.048137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.048645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.049073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.050114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.050226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.063942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.070710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af70e2-580e-495e-9949-2818e0e05ab1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.083851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.086007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.087796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.088097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.088369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.089086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.091079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.105267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 999/2025] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:54 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.110407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.147017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1027/2026] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.151998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.242478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.244416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30392a9-008c-485c-99bc-2ed969e674f1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.259702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bca3b-6832-47a7-84f6-864e4b56fb25 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.265518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.266334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.266841 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.267250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.268056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.270163 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.294840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1028/2027] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.298857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.311274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb23773-7fa8-4e27-8a15-cae21c33ddfc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.316339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.317068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.317359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.317666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.318442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.318739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.319551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.319875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.320181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.320296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.321530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.324274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.334685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1029/2028] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:55.334685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.336165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.348300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1000/2029] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/users => generated 290 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.349410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.357249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea73a49-8a57-44b3-b006-dd81f8bf1525 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.362869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.364025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.364281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.364566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] PATH_INFO: `/v3/projects/325e5218417547028ee4f19ff3020943` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.365768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=325e5218417547028ee4f19ff3020943)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.367758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.372159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccb320-32ce-40ef-905a-c07fd01e9312 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.376869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.377541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.377828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.378105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.378864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.380525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.390813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1001/2030] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:55.391676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.402155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f4bdb-ef77-4b7c-94d1-43342b771656 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.407654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.408417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.408778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.409187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.409943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.411973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.420445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '325e5218417547028ee4f19ff3020943', 'request_id': 'req-d358f966-3497-41cd-aacf-bc58eade43ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:55.422943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1002/2031] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:55.423756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.428661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '325e5218417547028ee4f19ff3020943', 'request_id': 'req-d358f966-3497-41cd-aacf-bc58eade43ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:55.434399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '325e5218417547028ee4f19ff3020943', 'request_id': 'req-d358f966-3497-41cd-aacf-bc58eade43ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:55.435034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ed953-643a-4c3b-9438-e462b4e3c9f4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.446265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] The token cache is being invalidate because project 325e5218417547028ee4f19ff3020943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:55.446669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:55.447459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:55.449139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1030/2032] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:55 2022] DELETE /identity/v3/projects/325e5218417547028ee4f19ff3020943 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:55.449992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.454836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.491753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.492713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.493001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.493282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] PATH_INFO: `/v3/projects/b52f00efedc34bfa84d6aae456ab429d/users/755016dabc364205b9e419ff6a0205bd/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.506092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.510165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52f00efedc34bfa84d6aae456ab429d, user_id=755016dabc364205b9e419ff6a0205bd, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.513409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726ec810-ae3f-499f-ab76-1184b9baec08 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.519173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358f966-3497-41cd-aacf-bc58eade43ef admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.524350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.525031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.525322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.525607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.526346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.527653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1003/2033] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:55 2022] PUT /identity/v3/projects/b52f00efedc34bfa84d6aae456ab429d/users/755016dabc364205b9e419ff6a0205bd/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:55.528115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a23bbc-2793-4dd2-89ee-96beb2ca1e87 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.529162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.529448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.529728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.539172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1031/2034] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:55.540537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893e13-52d6-48ef-9b63-1114d59bad86 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.540974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893e13-52d6-48ef-9b63-1114d59bad86 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.541263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe893e13-52d6-48ef-9b63-1114d59bad86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.548128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:55.549345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.570753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fe893e13-52d6-48ef-9b63-1114d59bad86 None None] MFA Rules not processed for user `755016dabc364205b9e419ff6a0205bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:55.572407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fe893e13-52d6-48ef-9b63-1114d59bad86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.612833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.613372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1004/2035] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:55.614196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.624330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fbeb4-d8a3-44ca-a16f-2f176cee7a80 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.630156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.631075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.631389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.631667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] PATH_INFO: `/v3/projects/860db2d4c86b4818b68c75dcbc53eb27/users/08fdc049f4944f2f9066abf21f720e19/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.647222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=860db2d4c86b4818b68c75dcbc53eb27, user_id=08fdc049f4944f2f9066abf21f720e19, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.650142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.650255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1032/2036] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:55.651471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b22916-ec16-4f4f-ae72-e9a2c8a7fb5b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.651827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b22916-ec16-4f4f-ae72-e9a2c8a7fb5b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.652168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b22916-ec16-4f4f-ae72-e9a2c8a7fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.665222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1005/2037] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:55 2022] PUT /identity/v3/projects/860db2d4c86b4818b68c75dcbc53eb27/users/08fdc049f4944f2f9066abf21f720e19/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:55.666151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.667827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.672085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e0b22916-ec16-4f4f-ae72-e9a2c8a7fb5b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:55.673256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e0b22916-ec16-4f4f-ae72-e9a2c8a7fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.732685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1033/2038] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:55.733976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.734263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.734485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221c01d-835d-4d23-b29c-e9fb71fe51fc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.734593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.739460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.740143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.740431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.740723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.741420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.743366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.761138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] MFA Rules not processed for user `08fdc049f4944f2f9066abf21f720e19`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:55.762350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:55.773469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1006/2039] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.773469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.824503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.834085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1034/2040] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:55.834085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.834637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7cd0e-4e94-4ddb-87f8-4a612fc9a76f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.840992 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.842101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.842403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.842684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.843466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:55.844220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.845844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.847683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4056912-f52a-4b3d-ae16-759b240cfc4a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.854763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.855986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.856357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.857822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.862118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1007/2041] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:55.862469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.862559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.862653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.872602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316430b-c2cb-41b6-831e-3b84f53bb816 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.877753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.878412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.878698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.878987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.879702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.881631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.882588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1035/2042] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/users => generated 297 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.883428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.894779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97971059-2f4f-4ab3-9548-e36bc45d785f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.901038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:55.901732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:55.902020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:55.902299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:55.902993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:55.905384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:55.914539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1008/2043] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.915493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.936579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1036/2044] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:55.936579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:55.960865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:55.978606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ebf90-2a4c-413a-b8a7-379004720128 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:55.996874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61cc6d-493e-4c56-8593-fd8cc7c33d03 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.001644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.002348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.002632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.002909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.003646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.005627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.025814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1037/2045] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:55 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:56.028496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.036075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.036770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.037056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.037332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.038068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.039256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13755fe2-469f-41a4-9c30-5b27383d614f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.039799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.044178 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.044834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.045119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.045396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.046082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.048082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.053487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1009/2046] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.053487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.061687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6806a7-d6fe-4fc6-8b89-d906282b9887 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.066609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.067298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.067584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.067860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.068587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.069122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1038/2047] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:56.069954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.070171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.081714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1010/2048] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.081968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7ed9c-0886-48f9-8d90-423477d2314a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.082940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.088552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.089471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.089764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.090058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] PATH_INFO: `/v3/projects/620d1058a162439197cff4c90df2491f/users/9ebe3472d4424d859888c04bd46b81f3/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.093523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7c1a9-ca04-49a6-84d0-55c315f22874 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.098184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.098836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.099152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.099442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.100160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.101760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.106553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620d1058a162439197cff4c90df2491f, user_id=9ebe3472d4424d859888c04bd46b81f3, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.109623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.112863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1011/2049] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.113756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.124665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e839d03-bdbb-4c0a-9aa0-52c355b9654c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.125560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1039/2050] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/620d1058a162439197cff4c90df2491f/users/9ebe3472d4424d859888c04bd46b81f3/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:56.126374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.170621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:56.184912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.185676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.185979 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.186243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.186973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.188530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0446e-6844-41f7-9aed-e457eded9989 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.188641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.193624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.194704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.195257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.195617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.196571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.198581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.202659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1012/2051] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.202659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.215323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1040/2052] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.217651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.220006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62eabc-5131-43fd-a081-79355426b1cb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.226787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.227508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.227793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.228072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.229015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.229245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750f7-8b47-42cb-8bbc-ad682c369f48 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.231072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.235511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.236746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.237096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.237481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] PATH_INFO: `/v3/projects/c252ad59efaf40dc8f8e5f4c6a30d582/users/e302ddc1d7e1410e9cbbf96c3b3579e6/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.246843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1013/2053] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.246843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.257445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c252ad59efaf40dc8f8e5f4c6a30d582, user_id=e302ddc1d7e1410e9cbbf96c3b3579e6, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.259407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2afd6-7d00-4661-a09d-509be310c67a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.265114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.266208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.266647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.267358 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] PATH_INFO: `/v3/projects/2aefe42fc2be4abaa3e9fb80fdc6006a/users/2481bb023ba64600b4a926faaa034a59/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.269628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.286883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aefe42fc2be4abaa3e9fb80fdc6006a, user_id=2481bb023ba64600b4a926faaa034a59, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.296801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1041/2054] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/c252ad59efaf40dc8f8e5f4c6a30d582/users/e302ddc1d7e1410e9cbbf96c3b3579e6/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:56.296801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.297088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573e824-f068-440a-ab30-5d010896315a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.311988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1014/2055] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/2aefe42fc2be4abaa3e9fb80fdc6006a/users/2481bb023ba64600b4a926faaa034a59/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:56.313343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb1a-d0ea-4c66-a4af-d06ab7f682b3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.313634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb1a-d0ea-4c66-a4af-d06ab7f682b3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.313944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeedb1a-d0ea-4c66-a4af-d06ab7f682b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.339690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9aeedb1a-d0ea-4c66-a4af-d06ab7f682b3 None None] MFA Rules not processed for user `e302ddc1d7e1410e9cbbf96c3b3579e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:56.340847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9aeedb1a-d0ea-4c66-a4af-d06ab7f682b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:56.348052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:56.367531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dafc9-013c-4aaf-a833-6a780c369d45 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.411300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1015/2056] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:56.412655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66248ab9-4b16-41c5-bb15-9e3311adb33e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.412947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66248ab9-4b16-41c5-bb15-9e3311adb33e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.413226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66248ab9-4b16-41c5-bb15-9e3311adb33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.424637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.425555 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.425892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.426177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] PATH_INFO: `/v3/projects/620d1058a162439197cff4c90df2491f/users/9ebe3472d4424d859888c04bd46b81f3/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.428190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620d1058a162439197cff4c90df2491f, user_id=9ebe3472d4424d859888c04bd46b81f3, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.430858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.431573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-66248ab9-4b16-41c5-bb15-9e3311adb33e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:56.432782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-66248ab9-4b16-41c5-bb15-9e3311adb33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:56.441290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1042/2057] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/620d1058a162439197cff4c90df2491f/users/9ebe3472d4424d859888c04bd46b81f3/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:56.441904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1016/2058] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:56.442412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.453539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.453829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.454134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.480628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] MFA Rules not processed for user `9ebe3472d4424d859888c04bd46b81f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:56.482524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:56.515017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6019aaca-c279-442c-9083-6e7e132da534 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.519987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.520699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.521099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.521436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.522223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.524372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.553557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1017/2059] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:56.554551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.555582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1043/2060] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:56.556132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:56.559446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.577614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:56.622374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4479fe-d760-40d9-80d8-6c2ecce03426 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.626497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d438e0-1d0f-4539-9bda-ba8b23b4c414 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.631330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.632009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.632389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.632719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.633505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.635556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.651037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:56.656928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1044/2061] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:56 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:56.657529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.659814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:56.677642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.678312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.678594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.678871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.688061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.689508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.699129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1018/2062] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:56.700009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.710506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f852d4b-b84b-41be-98c1-61fcf9cc3ba6 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.716378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.717145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.717447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.717722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.718515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.720307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.721463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f83dc9-93f9-4473-93f1-c407afe75ca7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.729786 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.730484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.730747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.731120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.735156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1019/2063] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:56.740091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.751740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0ec8f-5c0a-4344-ade4-1133743cf734 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.757783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.758333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.758623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.759867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] PATH_INFO: `/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2/users/a67397e522424dcdac23b5462b7ae5e2/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.779652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbdbbcc10dd4d6f9963c6d5e21f88e2, user_id=a67397e522424dcdac23b5462b7ae5e2, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.788517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.817135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.818968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.823309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1020/2064] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2/users/a67397e522424dcdac23b5462b7ae5e2/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:56.830198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.833859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1045/2065] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:56.939986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e7374-1cbc-4f07-9b7c-f019ed63b621 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.948376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:56.949284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:56.949699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:56.950133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:56.951401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:56.953444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:56.953609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:56.974514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc0e52-147f-4a28-bc54-6e3408707b78 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:56.995692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1021/2066] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:57.002660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.029929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef128cf3-f2c9-486a-835b-8dea4338db0e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:57.046141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.047175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.047525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.047873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] PATH_INFO: `/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2/users/a67397e522424dcdac23b5462b7ae5e2/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.053007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbdbbcc10dd4d6f9963c6d5e21f88e2, user_id=a67397e522424dcdac23b5462b7ae5e2, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:57.058663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56967c53-0002-4196-892e-d4d6281f7cd8 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:57.089131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.090099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.090541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.099620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1022/2067] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:56 2022] PUT /identity/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2/users/a67397e522424dcdac23b5462b7ae5e2/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:57.101613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc68562-d193-487d-9456-48785d87f422 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.101849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.101974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc68562-d193-487d-9456-48785d87f422 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.102267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc68562-d193-487d-9456-48785d87f422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.161131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4fc68562-d193-487d-9456-48785d87f422 None None] MFA Rules not processed for user `a67397e522424dcdac23b5462b7ae5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:57.163273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4fc68562-d193-487d-9456-48785d87f422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.184622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:57.202093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:57.204183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:57.223326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1046/2068] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:57.314594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.316708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.332278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:57.336669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1023/2069] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:57.340826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8d1e1-d5c0-456b-b029-9353e3f4fabf service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:57.440563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.442415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.443030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.443262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] PATH_INFO: `/v3/users/85302e62962f4d9ba8f95ea2b4c6b3ef` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.444540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc68562-d193-487d-9456-48785d87f422 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.445261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85302e62962f4d9ba8f95ea2b4c6b3ef)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:57.448238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:57.531180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85302e62962f4d9ba8f95ea2b4c6b3ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:57.546987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85302e62962f4d9ba8f95ea2b4c6b3ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:57.547232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc68562-d193-487d-9456-48785d87f422 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:57.554099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.554890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.557150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85302e62962f4d9ba8f95ea2b4c6b3ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:57.557334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85302e62962f4d9ba8f95ea2b4c6b3ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:57.558076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-20a29218-64d3-4857-8e5b-e2dcea6d3a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85302e62962f4d9ba8f95ea2b4c6b3ef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:57.558487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.558785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.560852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.582256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1047/2070] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:57 2022] DELETE /identity/v3/users/85302e62962f4d9ba8f95ea2b4c6b3ef => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:57.586874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.587759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.587759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.594195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:57.595779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3188db3-26ef-4731-9bdf-5d6fd0bea98a service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:57.609405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1024/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:57.612423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:57.614380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.721292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.721577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.721883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.726942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1048/2072] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:57.740713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.742460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.788530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] MFA Rules not processed for user `2481bb023ba64600b4a926faaa034a59`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:57.789646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:57.791341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.851677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b37c0b-01fa-4b35-96d3-3afdbb0993cd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:57.856133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.857064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.857401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.857748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.865566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.907212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1025/2073] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:57.908582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.915231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.941177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:57.942924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:57.960125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1049/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:15:57.961244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:57.962857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:57.984628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f9b60-5603-4f9f-bf38-8e45158fdc4f service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:57.991798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:57.992632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:57.993284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:57.993424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:57.994587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.000367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:58.003563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.027541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f680f-18ac-48d8-8690-9f6ed1356794 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.040575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.041318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.041660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.041986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.055735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1050/2075] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:57 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:58.061461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.062775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.066570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.083831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1026/2076] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:58.166481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbf7ba-d5f7-4ff8-b789-9b035ca1b143 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.173430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.191153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.191537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.191738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.192558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.195308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.230438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1051/2077] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:58 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:58.231946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.243600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaaaaf-f78b-4cc5-9332-4c8666569963 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.251222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.251222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.251641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.252057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.252929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.254916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.268531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1027/2078] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:15:58.273192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.287833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3c379-6fed-400e-8e08-463f49c2c7c4 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.295116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.296124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.296413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.297513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] PATH_INFO: `/v3/projects/bd62340ba4d9428f87bfef02a660ec8c/users/939a4c546e2a4b69a99b4135e2f9ada7/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.332179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd62340ba4d9428f87bfef02a660ec8c, user_id=939a4c546e2a4b69a99b4135e2f9ada7, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.339031 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.364374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1052/2079] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:58 2022] PUT /identity/v3/projects/bd62340ba4d9428f87bfef02a660ec8c/users/939a4c546e2a4b69a99b4135e2f9ada7/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:58.368248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.382185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bd4dc2-bb35-412a-b377-e300041b5642 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.474617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.475482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.475821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.476108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.477174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.478595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.495336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1028/2080] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:58.497064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.519706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ea973-b181-4086-a109-cb60f1178027 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.524565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.527555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:58.542998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.544320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.544753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.545171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] PATH_INFO: `/v3/projects/bd62340ba4d9428f87bfef02a660ec8c/users/939a4c546e2a4b69a99b4135e2f9ada7/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.547803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd62340ba4d9428f87bfef02a660ec8c, user_id=939a4c546e2a4b69a99b4135e2f9ada7, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.550906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad84157-34a5-443f-a184-c512e49e4b3b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.551726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2b251-7733-4297-8a67-31dd0f2b4244 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.556723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.557610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.557899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.558178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] PATH_INFO: `/v3/projects/3f9bce41dfc54cf6b5ef9cae58d8b753` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.559500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9bce41dfc54cf6b5ef9cae58d8b753)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.563000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.577298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1053/2081] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:58 2022] PUT /identity/v3/projects/bd62340ba4d9428f87bfef02a660ec8c/users/939a4c546e2a4b69a99b4135e2f9ada7/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:58.628185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9bce41dfc54cf6b5ef9cae58d8b753', 'request_id': 'req-7df715e8-9473-4963-8e4a-ed51300ea389'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:58.641966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9bce41dfc54cf6b5ef9cae58d8b753', 'request_id': 'req-7df715e8-9473-4963-8e4a-ed51300ea389'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:58.650584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9bce41dfc54cf6b5ef9cae58d8b753', 'request_id': 'req-7df715e8-9473-4963-8e4a-ed51300ea389'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:58.670230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] The token cache is being invalidate because project 3f9bce41dfc54cf6b5ef9cae58d8b753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:15:58.670687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:58.671482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:58.674366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1029/2082] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:58 2022] DELETE /identity/v3/projects/3f9bce41dfc54cf6b5ef9cae58d8b753 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:58.679245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70884-9e1f-4cb9-b33b-9cdb6b2a7d2a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.679650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70884-9e1f-4cb9-b33b-9cdb6b2a7d2a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.680459 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b70884-9e1f-4cb9-b33b-9cdb6b2a7d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.707877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-55b70884-9e1f-4cb9-b33b-9cdb6b2a7d2a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:58.709350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-55b70884-9e1f-4cb9-b33b-9cdb6b2a7d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:58.715572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:58.748066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.750865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:58.866098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1054/2083] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:58.869102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.869397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.869675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.879976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:58.908774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df715e8-9473-4963-8e4a-ed51300ea389 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.910640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:58.912660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:58.915211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.916330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.916839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.917411 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] PATH_INFO: `/v3/users/755016dabc364205b9e419ff6a0205bd` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.929585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1055/2084] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:15:58.929585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.930121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755016dabc364205b9e419ff6a0205bd)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.932779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.938105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80754c8-4e71-4c55-b539-a160f62d9949 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.943454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.944133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.944598 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.944847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.945849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:15:58.946354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.948834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:58.966677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1056/2085] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:15:58.971168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:58.984798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25368c59-d3b4-43e9-bffb-8760c419d451 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:58.992191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:58.992963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:58.993252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:58.993582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:58.994756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:58.998500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.014876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755016dabc364205b9e419ff6a0205bd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:59.031874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755016dabc364205b9e419ff6a0205bd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:59.036329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1057/2086] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:15:58 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:59.038733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.054975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755016dabc364205b9e419ff6a0205bd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:59.056267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755016dabc364205b9e419ff6a0205bd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:59.072803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c3b22647-7db0-4053-a0b1-cfb143260274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755016dabc364205b9e419ff6a0205bd'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:15:59.089826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1030/2087] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:15:58 2022] DELETE /identity/v3/users/755016dabc364205b9e419ff6a0205bd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:15:59.092273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.092789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.093130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.119430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:59.121754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.127084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1279b9-2cec-4b74-8d11-b9411af4a424 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.226237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.227884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.228138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.228436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.229239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.231248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.257352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1031/2088] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:59.265877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1058/2089] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:15:59 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:15:59.271194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.273739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.289805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd106a-2c88-44e9-82eb-040755db4dcf None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.295727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.296746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.297547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.297805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.298341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.298954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.299948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.301901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.308880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:59.330836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1032/2090] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:59.332093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.352666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe5be3-87f1-46e1-b605-6bcfb39f3ced admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.359315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.360484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.360777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.361063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.362178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.364557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.386327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1033/2091] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:15:59.389443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.400493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba0c8d-2def-4a14-bc58-f928c2c4105e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.405589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.406557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.409679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.410494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.412624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f01fc-1e0d-4252-b995-96e87107bb1b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.422700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.422700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.422700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.422700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] PATH_INFO: `/v3/projects/94100e07e4d34a679ae537cb87de4dba/users/787ef24488804250a58038c7378623ae/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.424325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.429034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27415d9f-4549-47b6-a49c-3be2712d7916 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.440657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=94100e07e4d34a679ae537cb87de4dba, user_id=787ef24488804250a58038c7378623ae, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.443636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1059/2092] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:59.444014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.462220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1034/2093] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:15:59 2022] PUT /identity/v3/projects/94100e07e4d34a679ae537cb87de4dba/users/787ef24488804250a58038c7378623ae/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:15:59.467359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1982bc1-b4c2-419e-b2ca-14bd21ead572 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.467656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1982bc1-b4c2-419e-b2ca-14bd21ead572 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.467943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1982bc1-b4c2-419e-b2ca-14bd21ead572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.491279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d1982bc1-b4c2-419e-b2ca-14bd21ead572 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:59.492888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d1982bc1-b4c2-419e-b2ca-14bd21ead572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.588551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1060/2094] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:15:59.628341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.631848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.736995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:15:59.813573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bdb65-02b4-4bb1-be08-889fea3d4cca admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.820201 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.821551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.822500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.822883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.833529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.833869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.834150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.854589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.855799 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:15:59.871918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1035/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:15:59.884565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:15:59.886545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.898570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] MFA Rules not processed for user `939a4c546e2a4b69a99b4135e2f9ada7`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:15:59.900214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:15:59.909605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865d1a73-6469-4b99-9e77-dcbaf5bd1414 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:15:59.918037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:15:59.921244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:15:59.921244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:15:59.923901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] PATH_INFO: `/v3/projects/b52f00efedc34bfa84d6aae456ab429d` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:15:59.926167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52f00efedc34bfa84d6aae456ab429d)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:15:59.930101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:00.037575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1061/2096] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:15:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:00.051930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52f00efedc34bfa84d6aae456ab429d', 'request_id': 'req-ed6b7967-5077-4613-bf50-4c0c213d8463'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.065892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:00.067831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.070060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52f00efedc34bfa84d6aae456ab429d', 'request_id': 'req-ed6b7967-5077-4613-bf50-4c0c213d8463'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.090151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52f00efedc34bfa84d6aae456ab429d', 'request_id': 'req-ed6b7967-5077-4613-bf50-4c0c213d8463'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.116153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] The token cache is being invalidate because project b52f00efedc34bfa84d6aae456ab429d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:00.117390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.118877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.125791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:00.126021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1036/2097] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:15:59 2022] DELETE /identity/v3/projects/b52f00efedc34bfa84d6aae456ab429d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:00.128140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:00.131580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.163670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6be24b-15f9-4642-85ca-4f4d606c9f93 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:00.246567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7967-5077-4613-bf50-4c0c213d8463 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:00.254295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:00.255069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:00.255351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:00.255729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:00.256046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:00.256288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:00.256671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:00.256894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] PATH_INFO: `/v3/users/9ebe3472d4424d859888c04bd46b81f3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:00.257989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.258329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebe3472d4424d859888c04bd46b81f3)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:00.260934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:00.333964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebe3472d4424d859888c04bd46b81f3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.347818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:00.349337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:00.355872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:00.361683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1062/2098] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:00.362919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebe3472d4424d859888c04bd46b81f3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.384924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebe3472d4424d859888c04bd46b81f3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.386046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebe3472d4424d859888c04bd46b81f3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.399169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-11e64f05-9fe5-42da-abb1-b178b5096092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebe3472d4424d859888c04bd46b81f3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.417418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1037/2099] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:00 2022] DELETE /identity/v3/users/9ebe3472d4424d859888c04bd46b81f3 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:00.476248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:00.478639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.507672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e0f70-92ba-42f5-af78-68846063aff7 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:00.605093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:00.606319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:00.606526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:00.607115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] PATH_INFO: `/v3/users/03f46752fcf64283881170d63c76f424` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:00.608495 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f46752fcf64283881170d63c76f424)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:00.611420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:00.711855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:00.715364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f46752fcf64283881170d63c76f424'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.747568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f46752fcf64283881170d63c76f424'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.767863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f46752fcf64283881170d63c76f424'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.769450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f46752fcf64283881170d63c76f424'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.770586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f46752fcf64283881170d63c76f424'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:00.799456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1063/2100] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:00 2022] DELETE /identity/v3/users/03f46752fcf64283881170d63c76f424 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:00.804401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc018a-70d8-439c-8a55-54482f4753ea None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:00.804678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc018a-70d8-439c-8a55-54482f4753ea None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:00.805028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc018a-70d8-439c-8a55-54482f4753ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:00.828200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:00.833375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.848395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-94dc018a-70d8-439c-8a55-54482f4753ea None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:00.851348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-94dc018a-70d8-439c-8a55-54482f4753ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:00.975964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf2c7-81c8-4c12-97f8-71eac6904a7e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:00.985322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:00.986332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:00.986713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:00.987197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] PATH_INFO: `/v3/projects/620d1058a162439197cff4c90df2491f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:00.988690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=620d1058a162439197cff4c90df2491f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:00.991196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.014814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1038/2101] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:01.019701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b193a-acca-4433-8e95-47e5c9a1259c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.021091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b193a-acca-4433-8e95-47e5c9a1259c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.023123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b193a-acca-4433-8e95-47e5c9a1259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.078400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620d1058a162439197cff4c90df2491f', 'request_id': 'req-58456ccf-a433-4be2-a11b-291eb2115337'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.090994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620d1058a162439197cff4c90df2491f', 'request_id': 'req-58456ccf-a433-4be2-a11b-291eb2115337'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.098361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-996b193a-acca-4433-8e95-47e5c9a1259c None None] MFA Rules not processed for user `787ef24488804250a58038c7378623ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:01.099646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-996b193a-acca-4433-8e95-47e5c9a1259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.109746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620d1058a162439197cff4c90df2491f', 'request_id': 'req-58456ccf-a433-4be2-a11b-291eb2115337'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.140455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] The token cache is being invalidate because project 620d1058a162439197cff4c90df2491f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:01.143278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.146719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.149104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1064/2102] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:00 2022] DELETE /identity/v3/projects/620d1058a162439197cff4c90df2491f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:01.149406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.156672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.210091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1039/2103] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:01.212383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.213061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.213550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.238015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:01.239271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.246028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58456ccf-a433-4be2-a11b-291eb2115337 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.252245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.252961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.253282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.253560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.255512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.263779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:01.300200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:01.310091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1040/2104] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:01.311009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.317658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.327836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.329290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.339831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1065/2105] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:01.340631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.341984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.353978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd78a8-abb1-4d73-97f7-0ae38302bf99 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.362088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.363159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.363539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.363866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] PATH_INFO: `/v3/users/bfa5c8210ead456ca337b3e54a0fb591` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.365357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa5c8210ead456ca337b3e54a0fb591)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.368503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.379815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9188b5-0ba1-46c6-9609-b6dd17a89c03 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.384402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.385087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.385376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.385657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.395863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.397373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db35dc-7a83-4a87-a83e-ff9521f0ba08 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.409171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1041/2106] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:01.413292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88801f-fe82-46da-88b0-52371802906c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.413645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88801f-fe82-46da-88b0-52371802906c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.414013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88801f-fe82-46da-88b0-52371802906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.444118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa5c8210ead456ca337b3e54a0fb591'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.460973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa5c8210ead456ca337b3e54a0fb591'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.468367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ea88801f-fe82-46da-88b0-52371802906c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:01.469506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa5c8210ead456ca337b3e54a0fb591'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.470828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa5c8210ead456ca337b3e54a0fb591'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.471510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a156785f-676a-488f-b783-f01773b5a59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa5c8210ead456ca337b3e54a0fb591'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.471888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ea88801f-fe82-46da-88b0-52371802906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.490454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1042/2107] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:01.494886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88801f-fe82-46da-88b0-52371802906c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.503349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1066/2108] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:01 2022] DELETE /identity/v3/users/bfa5c8210ead456ca337b3e54a0fb591 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:01.508535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.508892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.509124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.515259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88801f-fe82-46da-88b0-52371802906c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.541784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:01.545312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.596905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.597934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.598370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.598848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.599685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:01.600060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.602167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.618134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1043/2109] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:01.622043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.635730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496fc2c-554e-4a36-af8d-9a6de09b7004 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.645701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.647129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.647444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.647763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.648467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.650792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.675974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1067/2110] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:01.676886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.678546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.687257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1044/2111] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:01.689312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.696221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213a89-518e-49da-a997-0d2acd365b29 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.701633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.712472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:01.773846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.775030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.775224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec3c42-c9e5-4d11-bd7e-e31f8d8ec0e9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.775866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.776122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] PATH_INFO: `/v3/projects/d9abec1b410a4d089a04eb0024175b2e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.777543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9abec1b410a4d089a04eb0024175b2e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.780269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.780626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.781304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.781760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.782217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.801511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.803430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.815261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1045/2112] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:01.815929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.831963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4557-d786-449d-8c2d-74b08cbd4ab3 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.845623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:01.849480 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9abec1b410a4d089a04eb0024175b2e', 'request_id': 'req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.863529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9abec1b410a4d089a04eb0024175b2e', 'request_id': 'req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.871553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9abec1b410a4d089a04eb0024175b2e', 'request_id': 'req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.885859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] The token cache is being invalidate because project d9abec1b410a4d089a04eb0024175b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:01.886180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.887147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:01.892204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1068/2113] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:01 2022] DELETE /identity/v3/projects/d9abec1b410a4d089a04eb0024175b2e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:01.893154 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.894803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.925037 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.925754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.926051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.926565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.927041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.929426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.952380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1046/2114] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:01 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:01.956466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:01.959502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:01.972189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b30ddf-2f65-4c5e-8d9d-7e3dd46305f9 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.972646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb098e-2b0d-42be-84c8-6c1fd8f798de admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:01.977531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.978498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.978849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.978946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:01.979226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] PATH_INFO: `/v3/users/8d4f86fda9a24e7582f6ad1946eba93a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.979607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:01.979942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:01.980230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:01.980718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4f86fda9a24e7582f6ad1946eba93a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.981015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:01.982932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.983064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:01.994674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1047/2115] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:01.998625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.011565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548bb0a-623e-4e1a-b671-eefba64144bb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.018544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.019237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.019542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.019810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.020544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.022164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.037460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1048/2116] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:02.038401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.048977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271dcf-6070-47d7-aa18-35b037386e55 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.054560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.055621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.056014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.056336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] PATH_INFO: `/v3/projects/8278689e8d5146248c1730cd4a285e4a/users/29f3946dbb5440bfa063a833f7740a30/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.075400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4f86fda9a24e7582f6ad1946eba93a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.076446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8278689e8d5146248c1730cd4a285e4a, user_id=29f3946dbb5440bfa063a833f7740a30, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.076942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:02.079915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de4b10-594c-4578-9cc1-c71e7352bf34 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.103824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1049/2117] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:02 2022] PUT /identity/v3/projects/8278689e8d5146248c1730cd4a285e4a/users/29f3946dbb5440bfa063a833f7740a30/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:02.108231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4f86fda9a24e7582f6ad1946eba93a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.114650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.114945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.116021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.138506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4f86fda9a24e7582f6ad1946eba93a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.139761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4f86fda9a24e7582f6ad1946eba93a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.140527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4f86fda9a24e7582f6ad1946eba93a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.159139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] MFA Rules not processed for user `29f3946dbb5440bfa063a833f7740a30`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:02.160981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1069/2118] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:01 2022] DELETE /identity/v3/users/8d4f86fda9a24e7582f6ad1946eba93a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:02.161102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.161987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.163356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.277151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99075-ae91-4320-9910-46f9ce650fe4 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.277982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1050/2119] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:02.280300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.281509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.287562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.288118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.288542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.288866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.294560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.337425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:02.362216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14eb6f-8ab5-4f6b-a84f-16d3c5059252 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.369544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.370572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.370892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.371211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] PATH_INFO: `/v3/projects/1a39c978ab4d4266a39f6f9cc8cb1c34` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.372780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a39c978ab4d4266a39f6f9cc8cb1c34)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.375006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.384377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.385846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.406549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1070/2120] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:02.408320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.412107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.432894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664046ae-ec32-41cc-808d-f5bc92ba7f68 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.437283 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a39c978ab4d4266a39f6f9cc8cb1c34', 'request_id': 'req-c2044f68-9457-40e9-b67e-85080f18f75e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.452633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a39c978ab4d4266a39f6f9cc8cb1c34', 'request_id': 'req-c2044f68-9457-40e9-b67e-85080f18f75e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.463401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a39c978ab4d4266a39f6f9cc8cb1c34', 'request_id': 'req-c2044f68-9457-40e9-b67e-85080f18f75e'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.482657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] The token cache is being invalidate because project 1a39c978ab4d4266a39f6f9cc8cb1c34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:02.482981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.484360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.487035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1051/2121] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:02 2022] DELETE /identity/v3/projects/1a39c978ab4d4266a39f6f9cc8cb1c34 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:02.490713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.492338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.511811 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.515344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.515821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.516116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.517019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.519540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.547038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:02.556965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1071/2122] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:02 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:02.557874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.560035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.590371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2044f68-9457-40e9-b67e-85080f18f75e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.639333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93200b-9aa6-4cf4-8fcd-dada4eeb1aa7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.645572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.646486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.646852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.647053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] PATH_INFO: `/v3/users/4a03d71c08324b5e852d630c717f49a2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.648393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a03d71c08324b5e852d630c717f49a2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.650433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.654784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.655712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.655995 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.656275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] PATH_INFO: `/v3/users/08fdc049f4944f2f9066abf21f720e19` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.657504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08fdc049f4944f2f9066abf21f720e19)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.659612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.692048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:02.720729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a03d71c08324b5e852d630c717f49a2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.721367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08fdc049f4944f2f9066abf21f720e19'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.728427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08fdc049f4944f2f9066abf21f720e19'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.732939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a03d71c08324b5e852d630c717f49a2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.739833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a03d71c08324b5e852d630c717f49a2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.740531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a03d71c08324b5e852d630c717f49a2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.740710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08fdc049f4944f2f9066abf21f720e19'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.741203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a03d71c08324b5e852d630c717f49a2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.741389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08fdc049f4944f2f9066abf21f720e19'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.752644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08fdc049f4944f2f9066abf21f720e19'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:02.755645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1072/2123] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:02 2022] DELETE /identity/v3/users/4a03d71c08324b5e852d630c717f49a2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:02.758746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1052/2124] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:02 2022] DELETE /identity/v3/users/08fdc049f4944f2f9066abf21f720e19 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:02.761785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.761907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.761907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.762112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.832279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbeb33-9d32-45fd-bf7e-a8bbd9188bd8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.847628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daa01f-ee7e-4b5e-9fe6-9dd9abb05015 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.853083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.854084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.854348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.854762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.855598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.858324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d24ca-ad8a-4cb3-932c-a8d466639d0f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.864318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:02.885855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1053/2125] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:02 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:02.887469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.888216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.888981 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.910317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:02.911681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:02.915435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.916816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.917264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.917692 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] PATH_INFO: `/v3/projects/116e0db57f154ae1ba738e7f65529fff` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.919725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116e0db57f154ae1ba738e7f65529fff)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.922268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.929533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1054/2126] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:02.930634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:02.965378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830dacef-c6f4-4ba5-9dd1-909ccfba4c91 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:02.974487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:02.975354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.975645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.976101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:02.977025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:02.978704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4d500-e82d-44d5-be93-b832ae74e67b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:02.992112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1055/2127] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:02.993570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc95a-69b0-4426-b794-49286e599423 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:02.993862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc95a-69b0-4426-b794-49286e599423 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:02.994142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cc95a-69b0-4426-b794-49286e599423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.004691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116e0db57f154ae1ba738e7f65529fff', 'request_id': 'req-1d7db858-21c8-47ea-ab1e-907ec80076f1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.014226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116e0db57f154ae1ba738e7f65529fff', 'request_id': 'req-1d7db858-21c8-47ea-ab1e-907ec80076f1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.014973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-035cc95a-69b0-4426-b794-49286e599423 None None] MFA Rules not processed for user `787ef24488804250a58038c7378623ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.016649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-035cc95a-69b0-4426-b794-49286e599423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.021474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116e0db57f154ae1ba738e7f65529fff', 'request_id': 'req-1d7db858-21c8-47ea-ab1e-907ec80076f1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.033861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] The token cache is being invalidate because project 116e0db57f154ae1ba738e7f65529fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:03.034210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.035251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.037028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1073/2128] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:02 2022] DELETE /identity/v3/projects/116e0db57f154ae1ba738e7f65529fff => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:03.040824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.042145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.082322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1056/2129] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.083200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cc95a-69b0-4426-b794-49286e599423 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.084525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-035cc95a-69b0-4426-b794-49286e599423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.110009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7db858-21c8-47ea-ab1e-907ec80076f1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.119311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.120032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.120318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.120653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.122928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.141463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.205929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.207423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd31be-0578-45cd-ade1-e9f1cc0860fd service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.225270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cc95a-69b0-4426-b794-49286e599423 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.229609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.230038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1074/2130] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:03.230704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.231142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.231283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f43-261a-4149-97e1-42482b77946e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.231720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f43-261a-4149-97e1-42482b77946e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.231889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] PATH_INFO: `/v3/projects/6394e6538312426197aa05c04b3b3ca2/users/93c29e136f4243a5a00f96ef5f869b03/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.232008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f43-261a-4149-97e1-42482b77946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.247737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.248898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6394e6538312426197aa05c04b3b3ca2, user_id=93c29e136f4243a5a00f96ef5f869b03, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.251774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf0f8c-8b0f-40b3-99b2-c6cde97c0465 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.255872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-354c4f43-261a-4149-97e1-42482b77946e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.257138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-354c4f43-261a-4149-97e1-42482b77946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.267976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1075/2131] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.268854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4f43-261a-4149-97e1-42482b77946e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.270458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-354c4f43-261a-4149-97e1-42482b77946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.275701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1057/2132] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:03 2022] PUT /identity/v3/projects/6394e6538312426197aa05c04b3b3ca2/users/93c29e136f4243a5a00f96ef5f869b03/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:03.277170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.277463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.277739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.285621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4f43-261a-4149-97e1-42482b77946e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.298726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.300145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.345161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.346211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.346593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.346972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] PATH_INFO: `/v3/projects/c629663e83864ac6975587918803bc76` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.348219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c629663e83864ac6975587918803bc76)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.350151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.382218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1058/2133] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.383129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.395555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d155-b63e-473e-8315-3043cb45d674 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.413550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c629663e83864ac6975587918803bc76', 'request_id': 'req-50b56110-1b5d-41b8-912c-55ba32399c48'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.422229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c629663e83864ac6975587918803bc76', 'request_id': 'req-50b56110-1b5d-41b8-912c-55ba32399c48'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.429586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c629663e83864ac6975587918803bc76', 'request_id': 'req-50b56110-1b5d-41b8-912c-55ba32399c48'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.446158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] The token cache is being invalidate because project c629663e83864ac6975587918803bc76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:03.446468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.447213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.449177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1076/2134] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:03 2022] DELETE /identity/v3/projects/c629663e83864ac6975587918803bc76 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:03.453024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.454585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.461906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.462118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.462422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.462860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.463612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.465869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cddc0-55ad-497e-a40f-ed524ffdf0b4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.480426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1059/2135] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:03.480426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548463db-8a3f-4e0b-ac16-1c71f24dfb47 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.480426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548463db-8a3f-4e0b-ac16-1c71f24dfb47 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.480426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548463db-8a3f-4e0b-ac16-1c71f24dfb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.499252 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-548463db-8a3f-4e0b-ac16-1c71f24dfb47 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.500514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-548463db-8a3f-4e0b-ac16-1c71f24dfb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.510235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1060/2136] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.511910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.512202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.512676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.514558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1061/2137] 10.222.0.13 () {54 vars in 912 bytes} [Thu Jun 30 04:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:16:03.519728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.521648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.538498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b56110-1b5d-41b8-912c-55ba32399c48 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.543331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.544077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.544363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.544642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.545484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:03.545799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.547426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391f6bc-4cc7-4d3f-87e4-e0a318b736e8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.550220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354fea4-20c9-49db-89ab-3e1b200d5341 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.555289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.556315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.556702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.557095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] PATH_INFO: `/v3/projects/6394e6538312426197aa05c04b3b3ca2/users/93c29e136f4243a5a00f96ef5f869b03/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.559538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6394e6538312426197aa05c04b3b3ca2, user_id=93c29e136f4243a5a00f96ef5f869b03, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.561222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1077/2138] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:03.562377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.562665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.562949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.563196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3bd23-640d-4b14-97f1-6dd2a9938e85 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.582428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.583800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.592722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1062/2139] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:03 2022] PUT /identity/v3/projects/6394e6538312426197aa05c04b3b3ca2/users/93c29e136f4243a5a00f96ef5f869b03/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:03.594441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f038f9-e260-4bd5-9bae-1085181e5356 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.594728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f038f9-e260-4bd5-9bae-1085181e5356 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.595011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f038f9-e260-4bd5-9bae-1085181e5356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.604587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.616820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-86f038f9-e260-4bd5-9bae-1085181e5356 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.618286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-86f038f9-e260-4bd5-9bae-1085181e5356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.632533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.651754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1078/2140] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.652405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.663612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f2f-b815-45e3-90e1-a7598088a4f9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.668783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.669676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.670049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.670338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.671155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.673039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.691350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1063/2141] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.693105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.693505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.693797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.721487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1079/2142] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:03.722391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.731169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] MFA Rules not processed for user `93c29e136f4243a5a00f96ef5f869b03`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:03.732764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.794405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3557603-0c78-4c77-a83b-00e17f84bc01 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.799948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.800663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.800952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.801239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.801986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:03.802293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.803970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.813190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1064/2143] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:03.814114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.815457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.817068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1080/2144] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:03.818048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.819876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.829499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d9899-d7e5-4ff0-ab0c-db1231af93af None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.834159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.835058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.835369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.835652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] PATH_INFO: `/v3/projects/860db2d4c86b4818b68c75dcbc53eb27` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.836881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=860db2d4c86b4818b68c75dcbc53eb27)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.838819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.891411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8dc84f-206c-4b6e-9097-3b59bb2c444d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:03.892867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860db2d4c86b4818b68c75dcbc53eb27', 'request_id': 'req-204968c4-9501-483f-b54c-4a4bf89020dc'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.902114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860db2d4c86b4818b68c75dcbc53eb27', 'request_id': 'req-204968c4-9501-483f-b54c-4a4bf89020dc'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.909033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860db2d4c86b4818b68c75dcbc53eb27', 'request_id': 'req-204968c4-9501-483f-b54c-4a4bf89020dc'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.922441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] The token cache is being invalidate because project 860db2d4c86b4818b68c75dcbc53eb27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:03.922814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.923779 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:03.925570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1065/2145] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:03 2022] DELETE /identity/v3/projects/860db2d4c86b4818b68c75dcbc53eb27 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:03.926558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.928561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.949203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:03.949880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:03.950255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:03.950597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:03.952630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:03.960486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.963553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:03.965350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:03.976902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1081/2146] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:03.977823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:03.977966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:03.979360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.044794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204968c4-9501-483f-b54c-4a4bf89020dc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.057600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.058215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.058447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.058723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.059616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.061677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.102382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1066/2147] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/users => generated 282 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:04.102382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.102657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.110015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c928c1-5b78-4e86-95da-bfb463c99db1 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.115260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.116226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.116406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.116864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.117667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.119947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.149796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1082/2148] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:04.150987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.152731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.168421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460a293-9216-4ee8-85b8-9b2ae19a0637 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.230341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42c614-fcd6-4d07-9494-a414bb9e99f7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.235532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.236210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.236500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.237494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.238047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.238718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.239110 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.239773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] PATH_INFO: `/v3/users/e302ddc1d7e1410e9cbbf96c3b3579e6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.240354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.241218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e302ddc1d7e1410e9cbbf96c3b3579e6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.243521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.319516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.320824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e302ddc1d7e1410e9cbbf96c3b3579e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:04.320961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.332546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1067/2149] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:04.333728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.346702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aaead7-f394-4a32-b2a6-dbec842744bb service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.350565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e302ddc1d7e1410e9cbbf96c3b3579e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:04.352798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.353553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.353999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.354234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.355142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.357510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.360324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e302ddc1d7e1410e9cbbf96c3b3579e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:04.361747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e302ddc1d7e1410e9cbbf96c3b3579e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:04.363024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e302ddc1d7e1410e9cbbf96c3b3579e6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:04.377177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:04.382671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1083/2150] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:04 2022] DELETE /identity/v3/users/e302ddc1d7e1410e9cbbf96c3b3579e6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:04.383728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.388356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.392759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1068/2151] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:04.393983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.397152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.412357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c5e2-76cc-423a-b4d2-c1ef77c13c2f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.438225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:04.487471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c3a94-09f1-434b-af33-835236dfe5a7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.489221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.489919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.490215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.490492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.491269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.493091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.493470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.494009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.494404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.495021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.495854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.498177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.510006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1069/2152] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:04.511271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.527125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b573a54-8dba-491a-b39b-c9a2866b7ae9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.531214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.532127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.532416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.532697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] PATH_INFO: `/v3/projects/84ea90537ca545fc81b3720ba251ab71/users/630a7975886e408abe75f16ce2b4f4bf/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.540991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1084/2153] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:04 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:04.554509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.556525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ea90537ca545fc81b3720ba251ab71, user_id=630a7975886e408abe75f16ce2b4f4bf, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.559878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e280862-e023-4ec0-b94c-b5fcc49e9cd3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.574638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9d01f-fc6b-4e1d-8e8e-09e23287aa85 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.591695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.594214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.595790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.596828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.597935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.601619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1070/2154] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:04 2022] PUT /identity/v3/projects/84ea90537ca545fc81b3720ba251ab71/users/630a7975886e408abe75f16ce2b4f4bf/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:04.601806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.614870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.615517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.618582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.623877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1085/2155] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:04.624982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.640406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479f0f2-f43b-44fa-845f-71e75719d35d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.664041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] MFA Rules not processed for user `630a7975886e408abe75f16ce2b4f4bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:04.670674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.750938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.752569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.753075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.753525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.754737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.756977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.772759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1086/2156] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:04.778526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.796272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1071/2157] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:04.796877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b87da-8953-4ff3-8a27-ae45e8f89987 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.800918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.803520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.804685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.805319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.805929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] PATH_INFO: `/v3/projects/71bd18157d964712866540452b6ca95c/users/796f368f01de4e99bab83e01a976f81d/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.834442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bd18157d964712866540452b6ca95c, user_id=796f368f01de4e99bab83e01a976f81d, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.837064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a173c2b3-e8c6-46fe-87ae-aa6696639d66 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:04.840217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660525f-61a8-4b6b-9b86-c8778da46328 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.848971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:04.849907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.850280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.850643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.851542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:04.853601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:04.865511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1087/2158] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:04 2022] PUT /identity/v3/projects/71bd18157d964712866540452b6ca95c/users/796f368f01de4e99bab83e01a976f81d/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:04.878810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:04.879686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:04.881645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:04.942401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1072/2159] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:04 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:04.942401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:04.944511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:04.967657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] MFA Rules not processed for user `796f368f01de4e99bab83e01a976f81d`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:04.971202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.019379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.031862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edf5da-2f76-42fe-900c-4651f701652c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.037945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.038915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.039246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.039563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] PATH_INFO: `/v3/projects/c252ad59efaf40dc8f8e5f4c6a30d582` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.040950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c252ad59efaf40dc8f8e5f4c6a30d582)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.043284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.050420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1088/2160] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:05.051036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.053157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.064173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4afe0-720b-4bed-b99b-bb3fd61bd6d8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.070274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.070781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.072742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.100843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1089/2161] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:05 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:05.100843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.101361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c252ad59efaf40dc8f8e5f4c6a30d582', 'request_id': 'req-329e3a7b-35a8-4d18-b2e5-dadc505e179b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.108857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c252ad59efaf40dc8f8e5f4c6a30d582', 'request_id': 'req-329e3a7b-35a8-4d18-b2e5-dadc505e179b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.110003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916ef87-47c5-4eca-8bbd-7919ca36e63f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.116740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c252ad59efaf40dc8f8e5f4c6a30d582', 'request_id': 'req-329e3a7b-35a8-4d18-b2e5-dadc505e179b'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.129220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] The token cache is being invalidate because project c252ad59efaf40dc8f8e5f4c6a30d582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:05.129511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.131581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.134399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1073/2162] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:04 2022] DELETE /identity/v3/projects/c252ad59efaf40dc8f8e5f4c6a30d582 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:05.135320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.137102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.189286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.190101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.190448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.190760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.191501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.193470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.224020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1090/2163] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:05 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:05.224960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.228968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.235029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e3a7b-35a8-4d18-b2e5-dadc505e179b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.303001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ff1d8-4c5a-49db-98af-b6bf644abfdb admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.308710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.309802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.310549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.310886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] PATH_INFO: `/v3/users/a67397e522424dcdac23b5462b7ae5e2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.312846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67397e522424dcdac23b5462b7ae5e2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.313303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.314000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.314328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.314689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.315251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.315569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.317326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.334593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1074/2164] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:05.334593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.336042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.350167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b58401-4a10-4969-b3cd-eac679edc764 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.356482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.357789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.358290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.358643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.359566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.362807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.387127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67397e522424dcdac23b5462b7ae5e2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.388806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1075/2165] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:05 2022] POST /identity/v3/users => generated 294 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:05.389490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.392871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.401910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197263c-f555-4d48-97fb-004b17936135 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.404884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67397e522424dcdac23b5462b7ae5e2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.407483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.408386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.408675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.408958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] PATH_INFO: `/v3/projects/06739c5a7410476686556b679d82aa35/users/cecfbacf0f1147368fb7bec39f25b43a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.412093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67397e522424dcdac23b5462b7ae5e2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.412805 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67397e522424dcdac23b5462b7ae5e2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.413415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67397e522424dcdac23b5462b7ae5e2'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:05.425916 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=06739c5a7410476686556b679d82aa35, user_id=cecfbacf0f1147368fb7bec39f25b43a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.427749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1091/2166] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:05 2022] DELETE /identity/v3/users/a67397e522424dcdac23b5462b7ae5e2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:05.428635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.428959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.430065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.433244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.455725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1076/2167] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:05 2022] PUT /identity/v3/projects/06739c5a7410476686556b679d82aa35/users/cecfbacf0f1147368fb7bec39f25b43a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:05.462182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.464055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.520334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f07284-c9b4-48a0-a198-ef78dc5df51c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.544287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad430ec-dfd4-48ae-9379-656d03425cdf admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.551480 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.552257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.552956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.553330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.554521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.557040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.572700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1077/2168] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:05.578408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.599765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7ddeb-77b3-4a88-9273-f4386678c163 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.605433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.607490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.608390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.608851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] PATH_INFO: `/v3/projects/06739c5a7410476686556b679d82aa35/users/cecfbacf0f1147368fb7bec39f25b43a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.611560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=06739c5a7410476686556b679d82aa35, user_id=cecfbacf0f1147368fb7bec39f25b43a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.615363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8330b-4b25-4b0e-8945-77896bf6c16b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.631073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1078/2169] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:05 2022] PUT /identity/v3/projects/06739c5a7410476686556b679d82aa35/users/cecfbacf0f1147368fb7bec39f25b43a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:05.650454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.650454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.650454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.660747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.661707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.662165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.662589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.663503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.665302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.687856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1092/2170] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:05.687999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] MFA Rules not processed for user `cecfbacf0f1147368fb7bec39f25b43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:05.687999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.689090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.701334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada712e1-7342-422c-995a-38ea268d1bef admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.767626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.768433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.768745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.769107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.769759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.771543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.772719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1079/2171] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:05.785935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.787608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.793211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1093/2172] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:05.798231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.812248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701e3d6-bc9c-44ea-a59f-0811de96640b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.817096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.818187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.818539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.818876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] PATH_INFO: `/v3/projects/8c8c677a7b38452dac369e77de668006/users/026355dbea3843e1ad42eed6757a3404/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.828452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.848112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8c677a7b38452dac369e77de668006, user_id=026355dbea3843e1ad42eed6757a3404, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.850770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.860803 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9635517-ff3f-42e4-8bda-29f1dffbc472 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.865356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1094/2173] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:05 2022] PUT /identity/v3/projects/8c8c677a7b38452dac369e77de668006/users/026355dbea3843e1ad42eed6757a3404/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:05.866236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:05.867854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:05.881267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:05.918606 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.920057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.920057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.920341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.942199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae11dc3-6f18-430a-ac8d-47575eb9316f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:05.947141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:05.948060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:05.948490 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:05.948675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] PATH_INFO: `/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:05.949792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbdbbcc10dd4d6f9963c6d5e21f88e2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.952362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:05.990193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:05.992136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f34f8f-af3b-4f44-ac1c-8a5d8a3c2414 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.005440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1080/2174] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:06.006852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.007241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.007627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.022241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbdbbcc10dd4d6f9963c6d5e21f88e2', 'request_id': 'req-125562ab-2627-4792-a22a-f15c866ef20f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:06.039322 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] MFA Rules not processed for user `026355dbea3843e1ad42eed6757a3404`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:06.040824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.042171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbdbbcc10dd4d6f9963c6d5e21f88e2', 'request_id': 'req-125562ab-2627-4792-a22a-f15c866ef20f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:06.053327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbdbbcc10dd4d6f9963c6d5e21f88e2', 'request_id': 'req-125562ab-2627-4792-a22a-f15c866ef20f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:06.084595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] The token cache is being invalidate because project cbbdbbcc10dd4d6f9963c6d5e21f88e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:06.084901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:06.085626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-125562ab-2627-4792-a22a-f15c866ef20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:06.089930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1095/2175] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:05 2022] DELETE /identity/v3/projects/cbbdbbcc10dd4d6f9963c6d5e21f88e2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:06.096002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.096297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.096693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.120383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1081/2176] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:06.125091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.131519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.143883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:06.145123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.209265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1096/2177] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:06.209863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7445b-ef3a-42ec-a4ed-df07bb175e61 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.210002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.211298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.214399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.215062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.215394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.215757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.216932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.218959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63adf99-4a26-4492-b053-e807992c6ffb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.222869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:06.225603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:06.251087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1082/2178] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:06.251087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.251087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.251087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.268372 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:06.269604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.273431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96adc5b2-46ce-44aa-bb7e-e86910b0299c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.324517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.325296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.325633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.325914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.327749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.333573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1083/2179] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:06.333573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.341698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a11a74-d97d-45b2-82cf-e85d7b0c33ff None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.346548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.347502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.347648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.347845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.348531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.350594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.377236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1084/2180] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/users => generated 299 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:06.378273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.389248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468f789-7722-4cd3-b316-d2ea5531bd4d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.395618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.397042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.397220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.397748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.398027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.398713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.399048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:06.399396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.401350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.408571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1097/2181] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:06.409586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.420738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1085/2182] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:06.426541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.440845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f9a53-3c5f-4c0d-a2e1-ef45607ae8c8 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.445098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f64874-a405-46fa-acc5-b22183378cec admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.453306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.453306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.453306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.453306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.453664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.455269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.459899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.462033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.463533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.463818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.465580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.469200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.502092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1098/2183] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:06.503621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.524697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732511a0-f792-4c05-abe1-1fac725b9f80 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.526651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1086/2184] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:06.531186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.649911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87900784-aad2-4d37-a97f-d994a391d4a4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.656372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.657312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.657604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.657884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] PATH_INFO: `/v3/projects/900c14e3201840af90408b14e701e0d2/users/c7c99c50d8b54813a49a56032e34d93f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.658349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.659324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.661444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.661934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.662830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.665092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.675439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900c14e3201840af90408b14e701e0d2, user_id=c7c99c50d8b54813a49a56032e34d93f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.678481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.682513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:06.696965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1087/2185] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:06.701851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.703171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1099/2186] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:06 2022] PUT /identity/v3/projects/900c14e3201840af90408b14e701e0d2/users/c7c99c50d8b54813a49a56032e34d93f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:06.707422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.725259 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da41f63-dd4e-4393-8566-ab71361d4e35 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.757287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:06.790733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43f955-1993-41ff-a55b-839c9ebe0a54 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.799116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.799812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.800109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.800377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.801155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.803319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.809996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.810695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.810980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.811299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.812057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.814198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.814964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1100/2187] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:06.819164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.830536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b8d8d-5b15-42d0-ba0a-2f6216dc18ca admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.835594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:06.836738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.836848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1088/2188] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:06.836930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.837060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] PATH_INFO: `/v3/projects/900c14e3201840af90408b14e701e0d2/users/c7c99c50d8b54813a49a56032e34d93f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.839361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=900c14e3201840af90408b14e701e0d2, user_id=c7c99c50d8b54813a49a56032e34d93f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:06.841245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:06.842199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20930b-c052-479b-a7ca-8c270124cff8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:06.856715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a0187-255e-46f0-9944-295b0081c581 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:06.862656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1101/2189] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:06 2022] PUT /identity/v3/projects/900c14e3201840af90408b14e701e0d2/users/c7c99c50d8b54813a49a56032e34d93f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:06.864309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:06.864607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:06.864891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:06.895514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] MFA Rules not processed for user `c7c99c50d8b54813a49a56032e34d93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:06.896751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:06.989473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1102/2190] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:07.004293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.004530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.005422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.005630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.005913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.006674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.007043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.010175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.021628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1089/2191] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:07.025492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.036495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c800f-5db0-4e4d-9f5a-5aa0e058c489 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.043171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.044146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.044436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.044713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] PATH_INFO: `/v3/projects/6339d1a011b24bf0b5fbe559ac10e22e/users/9ae197f88e354bcc84dc655eb0b6b251/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.062188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6339d1a011b24bf0b5fbe559ac10e22e, user_id=9ae197f88e354bcc84dc655eb0b6b251, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.064873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a3fd8-9cbc-46c9-b85d-19a82d685baa admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.081135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1090/2192] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:07 2022] PUT /identity/v3/projects/6339d1a011b24bf0b5fbe559ac10e22e/users/9ae197f88e354bcc84dc655eb0b6b251/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:07.094937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.095333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.095618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.105308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78824625-1ed4-4fbe-b08e-385302233041 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.124866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] MFA Rules not processed for user `9ae197f88e354bcc84dc655eb0b6b251`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:07.126102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.128157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:07.165570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.166318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.166527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.166797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.225346 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1091/2193] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:07.226019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.227390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.244688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.246344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.250931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:07.257058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1103/2194] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:07.257627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.318327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed22066-c82d-4c36-9667-9bdc874d6192 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.325438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.326982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.327306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.327626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] PATH_INFO: `/v3/users/29f3946dbb5440bfa063a833f7740a30` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.329742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f3946dbb5440bfa063a833f7740a30)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.332395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.349058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e9724a-1176-415e-8318-8bb6375473cb service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.356508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.357334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.357625 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.358067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.359168 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.362164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.395987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1104/2195] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:07 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:07.400037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.409609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f3946dbb5440bfa063a833f7740a30'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:07.420019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f3946dbb5440bfa063a833f7740a30'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:07.433794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f3946dbb5440bfa063a833f7740a30'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:07.434710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f3946dbb5440bfa063a833f7740a30'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:07.444818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cdd8565c-ee07-4036-a90c-d4452b02a86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f3946dbb5440bfa063a833f7740a30'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:07.454374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1092/2196] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:07 2022] DELETE /identity/v3/users/29f3946dbb5440bfa063a833f7740a30 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:07.461840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.462208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.462644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.486679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:07.486993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:07.488079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.513230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a68d3-1623-49f5-ba7b-9bc05d955aca admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.612495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1093/2197] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:07.636013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.636788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.637074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.637355 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.639223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.641236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.643914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.646519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.677057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1105/2198] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:07 2022] POST /identity/v3/users => generated 296 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:07.682188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.685809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:07.705839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:07.720503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d97c96-b07a-406e-b891-705b7e902e6f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.730433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33bd90-8ca1-434d-9719-599f1e88210d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.735738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.736431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.736731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.736979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.737719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.739774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.756777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.757464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.757743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.758017 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.789158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1106/2199] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:07.790095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.790572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.792419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.813202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1094/2200] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:16:07.821859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62172ef2-f682-4af8-b1eb-b32cc207afa1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:07.845265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:07.846469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:07.846932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:07.847388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] PATH_INFO: `/v3/projects/d9387742c75743d5b09ac93692515658/users/0c0287bec9824d049725975789d7af67/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:07.881245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9387742c75743d5b09ac93692515658, user_id=0c0287bec9824d049725975789d7af67, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:07.884311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:07.914421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1107/2201] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:07 2022] PUT /identity/v3/projects/d9387742c75743d5b09ac93692515658/users/0c0287bec9824d049725975789d7af67/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:07.918430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:07.933128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68715bce-a7c0-449c-8d4f-cd07900600cb service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.110384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.112034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.112370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.113949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.115862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.118216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffc6a6-1c90-476d-89e8-8f84ebe6f87f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:08.137005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1095/2202] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:08.140890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:08.155029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bfa-936e-4c31-8055-b613ac606c89 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.162405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.163574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.164301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.164831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] PATH_INFO: `/v3/projects/d9387742c75743d5b09ac93692515658/users/0c0287bec9824d049725975789d7af67/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.169036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9387742c75743d5b09ac93692515658, user_id=0c0287bec9824d049725975789d7af67, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.173220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:08.196583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1108/2203] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:08 2022] PUT /identity/v3/projects/d9387742c75743d5b09ac93692515658/users/0c0287bec9824d049725975789d7af67/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:08.210373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.210749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.211199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.254776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] MFA Rules not processed for user `0c0287bec9824d049725975789d7af67`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:08.259292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:08.460531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1096/2204] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:08.520216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:08.521944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:08.584624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:08.586188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:08.608062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b09f36-ac3a-465c-a3f1-ebb6d616ea31 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.611910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.612614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.612938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.613221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.625146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.626941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:08.637705 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1109/2205] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:16:08.638547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:08.639941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:08.655000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e070f-a5c2-4a4a-a5a3-312f7b652348 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.660233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.660816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.661076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.661458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.663236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:08.688506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:08.702344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:08.723016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473e65b-f9f8-4441-9377-42e9906f5f36 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.733846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.734880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.735192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.735475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] PATH_INFO: `/v3/projects/8278689e8d5146248c1730cd4a285e4a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.736698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8278689e8d5146248c1730cd4a285e4a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.738568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:08.743823 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.745275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:08.756958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1097/2206] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:16:08.794829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8278689e8d5146248c1730cd4a285e4a', 'request_id': 'req-048d57a2-c828-40ad-bfbd-a5621bfcf024'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:08.806937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8278689e8d5146248c1730cd4a285e4a', 'request_id': 'req-048d57a2-c828-40ad-bfbd-a5621bfcf024'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:08.813738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8278689e8d5146248c1730cd4a285e4a', 'request_id': 'req-048d57a2-c828-40ad-bfbd-a5621bfcf024'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:08.827754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] The token cache is being invalidate because project 8278689e8d5146248c1730cd4a285e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:08.827754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:08.829091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:08.831157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1110/2207] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:08 2022] DELETE /identity/v3/projects/8278689e8d5146248c1730cd4a285e4a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:08.834962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:08.846904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d4976-9edf-4ec0-927b-48a54b0991ed service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:08.924488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:08.925490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:08.925786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:08.926183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] PATH_INFO: `/v3/users/93c29e136f4243a5a00f96ef5f869b03` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:08.928775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c29e136f4243a5a00f96ef5f869b03)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:08.929776 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.000009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c29e136f4243a5a00f96ef5f869b03'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.014102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c29e136f4243a5a00f96ef5f869b03'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.038350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c29e136f4243a5a00f96ef5f869b03'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.040036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c29e136f4243a5a00f96ef5f869b03'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.051602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-586ed5bc-bca0-4dc5-bebe-e58ad33f02e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c29e136f4243a5a00f96ef5f869b03'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.060882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1098/2208] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:08 2022] DELETE /identity/v3/users/93c29e136f4243a5a00f96ef5f869b03 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:09.125369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:09.129980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.149344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d57a2-c828-40ad-bfbd-a5621bfcf024 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:09.232758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:09.233809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.234098 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.234378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] PATH_INFO: `/v3/users/9ae197f88e354bcc84dc655eb0b6b251` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.235653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae197f88e354bcc84dc655eb0b6b251)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:09.237988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.293491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae197f88e354bcc84dc655eb0b6b251'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.306160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae197f88e354bcc84dc655eb0b6b251'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.313726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae197f88e354bcc84dc655eb0b6b251'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.314425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae197f88e354bcc84dc655eb0b6b251'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.315035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae197f88e354bcc84dc655eb0b6b251'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.318726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:09.331933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1111/2209] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:09 2022] DELETE /identity/v3/users/9ae197f88e354bcc84dc655eb0b6b251 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:09.336894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.337327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.337632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.356297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:09.358374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.405553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:09.408352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.408860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:09.473378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1099/2210] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:09.503428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:09.505997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.531772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3de0-e0f0-4f06-a052-9eea43548368 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:09.538935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:09.542942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.543365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.544753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] PATH_INFO: `/v3/projects/6394e6538312426197aa05c04b3b3ca2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.546508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6394e6538312426197aa05c04b3b3ca2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:09.549024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.584146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2fb18-17d7-42cb-a8c3-3a89c9c8e95e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:09.598889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:09.601112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.601676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.604478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.629184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:09.630865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89a18-5446-4d5a-a51f-2e78b84b627a service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.637214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6394e6538312426197aa05c04b3b3ca2', 'request_id': 'req-24fb1f6c-a580-4b2a-a87d-f97d5897a378'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.650123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6394e6538312426197aa05c04b3b3ca2', 'request_id': 'req-24fb1f6c-a580-4b2a-a87d-f97d5897a378'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.658568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6394e6538312426197aa05c04b3b3ca2', 'request_id': 'req-24fb1f6c-a580-4b2a-a87d-f97d5897a378'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.660808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1100/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:16:09.675246 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] The token cache is being invalidate because project 6394e6538312426197aa05c04b3b3ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:09.675576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.676807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-24fb1f6c-a580-4b2a-a87d-f97d5897a378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:09.679047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1112/2212] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:09 2022] DELETE /identity/v3/projects/6394e6538312426197aa05c04b3b3ca2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:09.685539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.686264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.686462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.710347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:09.711985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.811059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1101/2213] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:09.819691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.820216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.820741 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.849179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:09.851242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:09.862544 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1113/2214] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:09.867400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:09.880489 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ae510-a4ee-406c-aba7-30a6eba4cbcd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:09.886182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:09.887026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.887334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.887739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.888603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:09.888984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:09.891479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.911484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1102/2215] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:16:09.916118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:09.928262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb043fa-5eb1-4584-b9c8-d63481581ce9 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:09.937001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:09.937579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:09.937869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:09.938160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:09.939915 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:09.943080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:09.991139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1114/2216] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:09 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:09.995677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.009267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7163ff-93d6-4b3c-8f16-7fa9b50aed7f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.099922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.101011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.101656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.102209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.103200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.105320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.127715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:10.147078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1103/2217] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:09 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:10.149132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.182340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b19b9f-1575-4100-9652-64769407a1ba admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.214280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.229683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.229984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.230241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.230992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.247183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.290947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1115/2218] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:10.295832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.315389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2a537-93e8-4a78-8085-5c4fe8be91d5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.334061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.334768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.338664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.338965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.340170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.342902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb714520-a11f-422f-bb72-a731fd7933d2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.380645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1104/2219] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:10.387337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.402070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feacb6-dffb-4849-93cb-5973025b7f4b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.415742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.417103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.417562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.418020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] PATH_INFO: `/v3/projects/cab9a222afb642f1be0b056bd13dd203/users/5609e3e8016641c89572ee0db9aa3e81/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.454601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab9a222afb642f1be0b056bd13dd203, user_id=5609e3e8016641c89572ee0db9aa3e81, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.457567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.490633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1116/2220] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:10 2022] PUT /identity/v3/projects/cab9a222afb642f1be0b056bd13dd203/users/5609e3e8016641c89572ee0db9aa3e81/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:10.502841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459691-deca-442b-8fd5-559945da7334 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.503399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459691-deca-442b-8fd5-559945da7334 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.503477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459691-deca-442b-8fd5-559945da7334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.550874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.555410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e7459691-deca-442b-8fd5-559945da7334 None None] MFA Rules not processed for user `5609e3e8016641c89572ee0db9aa3e81`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:10.557032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e7459691-deca-442b-8fd5-559945da7334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:10.561439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:10.627982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:10.705290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1105/2221] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:10.730798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeec7-ca33-477b-a57d-bc0c703bb004 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.731744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459691-deca-442b-8fd5-559945da7334 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.733643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e7459691-deca-442b-8fd5-559945da7334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:10.736013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.736983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.737240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.737534 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] PATH_INFO: `/v3/projects/6339d1a011b24bf0b5fbe559ac10e22e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.738755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6339d1a011b24bf0b5fbe559ac10e22e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.740759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.807975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459691-deca-442b-8fd5-559945da7334 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.811473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6339d1a011b24bf0b5fbe559ac10e22e', 'request_id': 'req-b2ee8f0c-19d8-4aae-91d2-134430674900'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:10.823581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6339d1a011b24bf0b5fbe559ac10e22e', 'request_id': 'req-b2ee8f0c-19d8-4aae-91d2-134430674900'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:10.832918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6339d1a011b24bf0b5fbe559ac10e22e', 'request_id': 'req-b2ee8f0c-19d8-4aae-91d2-134430674900'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:10.846199 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] The token cache is being invalidate because project 6339d1a011b24bf0b5fbe559ac10e22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:10.846689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:10.847427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:10.849226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1117/2222] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:10 2022] DELETE /identity/v3/projects/6339d1a011b24bf0b5fbe559ac10e22e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:10.852833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.854205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:10.873955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.874677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.874974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.875291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.877960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:10.933159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:10.940806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee8f0c-19d8-4aae-91d2-134430674900 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:10.946020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:10.947124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:10.947432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:10.947763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] PATH_INFO: `/v3/users/0c0287bec9824d049725975789d7af67` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:10.949044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0287bec9824d049725975789d7af67)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.951411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.954354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:10.956186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:10.967511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1106/2223] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:10.992428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:10.994108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.009398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc13843-31c4-498c-a0fe-4f5393b4faa3 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.015042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.015927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.016339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.016506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] PATH_INFO: `/v3/users/5609e3e8016641c89572ee0db9aa3e81` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.017756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5609e3e8016641c89572ee0db9aa3e81)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.019762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.026979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0287bec9824d049725975789d7af67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.035084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:11.047475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0287bec9824d049725975789d7af67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.055611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0287bec9824d049725975789d7af67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.056577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0287bec9824d049725975789d7af67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.057344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-ba1ec187-9b39-44d2-9907-12b204cccaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0287bec9824d049725975789d7af67'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.072288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1118/2224] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:10 2022] DELETE /identity/v3/users/0c0287bec9824d049725975789d7af67 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:11.081911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5609e3e8016641c89572ee0db9aa3e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.089733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5609e3e8016641c89572ee0db9aa3e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.105772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5609e3e8016641c89572ee0db9aa3e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.106771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5609e3e8016641c89572ee0db9aa3e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.118076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5609e3e8016641c89572ee0db9aa3e81'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.125794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1107/2225] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:10 2022] DELETE /identity/v3/users/5609e3e8016641c89572ee0db9aa3e81 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:11.130373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.130772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.130977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.153985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:11.155574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.223719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1119/2226] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:11.242543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.242543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.325592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10623c0-8db5-43ac-9e81-8c99ce60d8d6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.330300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.331110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.331515 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.331864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.345204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.346706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773a828-49f2-46e2-87ce-5527eeb686db service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.359185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1108/2227] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:11.414739 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.416562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.430116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5377f-5b72-4f09-89fa-ade961e8e44d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.436263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.437237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.437638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.437852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] PATH_INFO: `/v3/projects/d9387742c75743d5b09ac93692515658` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.439455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9387742c75743d5b09ac93692515658)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.441624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.568299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9387742c75743d5b09ac93692515658', 'request_id': 'req-d007cede-ccf1-446b-acdf-28be3dba8fa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.595879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9387742c75743d5b09ac93692515658', 'request_id': 'req-d007cede-ccf1-446b-acdf-28be3dba8fa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.602989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9387742c75743d5b09ac93692515658', 'request_id': 'req-d007cede-ccf1-446b-acdf-28be3dba8fa1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.631067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] The token cache is being invalidate because project d9387742c75743d5b09ac93692515658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:11.631408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.632101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.633991 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1120/2228] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:11 2022] DELETE /identity/v3/projects/d9387742c75743d5b09ac93692515658 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:11.639887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1d4a-8736-4a11-9e93-988f29fdcec8 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.640400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1d4a-8736-4a11-9e93-988f29fdcec8 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.640509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1d4a-8736-4a11-9e93-988f29fdcec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.661232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7fcc1d4a-8736-4a11-9e93-988f29fdcec8 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:11.662578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7fcc1d4a-8736-4a11-9e93-988f29fdcec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.744343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1109/2229] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:11.747697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.752613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.762732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.763028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.763337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.772570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007cede-ccf1-446b-acdf-28be3dba8fa1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.774835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:11.778776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.779995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.780284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.780667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] PATH_INFO: `/v3/projects/cab9a222afb642f1be0b056bd13dd203` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.781958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab9a222afb642f1be0b056bd13dd203)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.784367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.799583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:11.801644 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.814926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1110/2230] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:11.816168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.828229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdee97c-1420-420e-989e-04d9b5cfe273 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.843312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab9a222afb642f1be0b056bd13dd203', 'request_id': 'req-7730398e-6676-4b1a-9283-40d192f389b0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.845101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:11.852596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab9a222afb642f1be0b056bd13dd203', 'request_id': 'req-7730398e-6676-4b1a-9283-40d192f389b0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.859349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab9a222afb642f1be0b056bd13dd203', 'request_id': 'req-7730398e-6676-4b1a-9283-40d192f389b0'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.870762 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] The token cache is being invalidate because project cab9a222afb642f1be0b056bd13dd203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:11.871050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.871731 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:11.873440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1121/2231] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:11 2022] DELETE /identity/v3/projects/cab9a222afb642f1be0b056bd13dd203 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:11.887739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.888521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.888743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.889011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.889800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:11.890112 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.891839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.910777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1111/2232] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:11.912391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.914036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:11.928565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730398e-6676-4b1a-9283-40d192f389b0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:11.933526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:11.934232 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:11.934575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:11.934893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:11.935606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:11.937540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:11.968855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1122/2233] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:11 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:11.973412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:11.984471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7a57-66fd-4b2f-924d-d5fc3e370532 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.048465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.049166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.049455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.049743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.050458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.052444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.076497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1112/2234] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:11 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:12.080745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.091847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4229b-e3c9-47d1-b522-e71011b3db1f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.096509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.097179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.097454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.097732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.098467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.100161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.110223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1123/2235] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:12.114728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.125649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3ba0d-afa5-4319-bb61-505ceb66085b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.131144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.131819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.132096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.132373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.133103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.134708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5998ff-40f9-48bc-b718-aed517662987 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.145102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1113/2236] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:12.149152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.159063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f53e91-1c3f-491d-a886-f2dac155e2f4 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.164097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.164980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.165260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.165531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] PATH_INFO: `/v3/projects/e31ab1240bb2431e9cae6b1588d409e2/users/117bbd1474aa416eab2de79761c4b28c/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.179680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31ab1240bb2431e9cae6b1588d409e2, user_id=117bbd1474aa416eab2de79761c4b28c, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.182359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.198193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1124/2237] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:12 2022] PUT /identity/v3/projects/e31ab1240bb2431e9cae6b1588d409e2/users/117bbd1474aa416eab2de79761c4b28c/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:12.208430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.208732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.209026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.236942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] MFA Rules not processed for user `117bbd1474aa416eab2de79761c4b28c`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:12.238186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:12.301025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:12.307148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1114/2238] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:12.312852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.402409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce050f-ad8d-4f2a-b700-7fa82fcbe658 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.408122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.408777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.409193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.409475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.410522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.412466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.450416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1125/2239] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:12 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:12.451683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.462773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4101e-4d37-4fd8-9494-bd2cc804b307 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.524081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.524663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.524950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.525233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.525934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.528012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.549574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1115/2240] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:12 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:12.553713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.564310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5de5dc-fa5a-404f-a24d-e937e3741c25 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.569431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.570139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.570395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.570673 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.571626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.573255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.584793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1126/2241] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:12.588866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.599927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39c5e4-2a70-4a08-aadd-60bf2eaeddcf admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.607080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.608026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.608302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.608610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] PATH_INFO: `/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e/users/2f736ec69e7d43359a166a5f91e3c24a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.624613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a0dcb352f848548c9ca2fa2b4d721e, user_id=2f736ec69e7d43359a166a5f91e3c24a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.627584 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.642542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1116/2242] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:12 2022] PUT /identity/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e/users/2f736ec69e7d43359a166a5f91e3c24a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:12.646642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.657120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad593c-9382-4ceb-a995-92683ddbd097 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.709566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:12.717740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.718517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.719206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.719504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.720397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.722418 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e362dc-8b0f-46fc-acee-f64d0b1dfad4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.732827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1127/2243] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:12.737178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.748872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde25fd2-9920-4751-a84b-c623d287d0f1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.754362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.755291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.755576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.755855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] PATH_INFO: `/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e/users/2f736ec69e7d43359a166a5f91e3c24a/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.758098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a0dcb352f848548c9ca2fa2b4d721e, user_id=2f736ec69e7d43359a166a5f91e3c24a, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:12.760818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:12.772579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1117/2244] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:12 2022] PUT /identity/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e/users/2f736ec69e7d43359a166a5f91e3c24a/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:12.785339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.785545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.785842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:12.813995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] MFA Rules not processed for user `2f736ec69e7d43359a166a5f91e3c24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:12.815255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:12.894354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1128/2245] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:12.908156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:12.910501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:12.971898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e01402-987f-4072-b293-58e1f153266e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:12.975964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:12.976612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:12.976913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:12.977162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:13.035416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:13.036795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:13.045631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1118/2246] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:16.518306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:16.520120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:16.534105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e74f15-a417-4dd8-8106-3024a3db392d None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:16.539604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:16.540967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:16.541296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:16.541600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:16.543759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:16.612557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:16.614187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:16.625196 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1129/2247] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:17.368623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:17.374829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:17.382728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15933a78-5180-49d6-85a4-731b600e367b service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:17.386656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:17.387398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:17.387886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:17.388172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:17.398458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:17.399943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bce1a1-f89b-4149-bae1-0d10db93eb44 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:17.413428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1119/2248] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:18.062213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:18.066378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:18.161026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca58ed-06a8-4ece-bb89-17dc49b345ab service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:18.168737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:18.170105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:18.170502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:18.170953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] PATH_INFO: `/v3/users/630a7975886e408abe75f16ce2b4f4bf` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:18.173229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630a7975886e408abe75f16ce2b4f4bf)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:18.184062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:18.213880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:18.268149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630a7975886e408abe75f16ce2b4f4bf'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:18.291814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630a7975886e408abe75f16ce2b4f4bf'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:18.307479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630a7975886e408abe75f16ce2b4f4bf'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:18.308587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630a7975886e408abe75f16ce2b4f4bf'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:18.309604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630a7975886e408abe75f16ce2b4f4bf'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:18.334030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1130/2249] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:18 2022] DELETE /identity/v3/users/630a7975886e408abe75f16ce2b4f4bf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:18.340534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:18.343576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:18.345838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:18.379968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:18.383901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:18.516868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1120/2250] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:18.529432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:18.531035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:18.629766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4afea-3630-4755-a001-d12f33d17146 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:18.633714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:18.634570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:18.634999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:18.635457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:18.651144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:18.652760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:18.664125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1131/2251] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:19.001698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:19.003277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:19.016761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:19.018649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:19.024466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055590f-9c7f-4a92-807d-88e5ed49e807 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:19.037944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b0e4d-1254-48f5-8023-07ef90969afe service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:19.040521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:19.041626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:19.041985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:19.042432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] PATH_INFO: `/v3/projects/84ea90537ca545fc81b3720ba251ab71` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:19.042775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:19.043750 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:19.044114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:19.044390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] PATH_INFO: `/v3/users/796f368f01de4e99bab83e01a976f81d` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:19.044980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ea90537ca545fc81b3720ba251ab71)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:19.045680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796f368f01de4e99bab83e01a976f81d)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:19.048430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:19.049750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:19.131684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796f368f01de4e99bab83e01a976f81d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.132627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ea90537ca545fc81b3720ba251ab71', 'request_id': 'req-f5fb9632-4442-4e9e-82d7-b440486f5ec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.148036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ea90537ca545fc81b3720ba251ab71', 'request_id': 'req-f5fb9632-4442-4e9e-82d7-b440486f5ec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.160211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ea90537ca545fc81b3720ba251ab71', 'request_id': 'req-f5fb9632-4442-4e9e-82d7-b440486f5ec3'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.160592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796f368f01de4e99bab83e01a976f81d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.174835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796f368f01de4e99bab83e01a976f81d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.176590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796f368f01de4e99bab83e01a976f81d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.177817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8e400a45-0b5a-45fd-8f90-897f8cde91c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796f368f01de4e99bab83e01a976f81d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.178177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] The token cache is being invalidate because project 84ea90537ca545fc81b3720ba251ab71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:19.178747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.179702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.182244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1121/2252] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:19 2022] DELETE /identity/v3/projects/84ea90537ca545fc81b3720ba251ab71 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:19.186418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:19.188885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:19.204073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1132/2253] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:19 2022] DELETE /identity/v3/users/796f368f01de4e99bab83e01a976f81d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:19.208529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:19.208999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:19.209421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:19.245392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:19.248341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:19.257698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:19.278922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb9632-4442-4e9e-82d7-b440486f5ec3 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:19.301815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:19.349794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1133/2254] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:19.356275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:19.357373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:19.357761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:19.358328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] PATH_INFO: `/v3/users/cecfbacf0f1147368fb7bec39f25b43a` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:19.361857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:19.363392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:19.363508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=cecfbacf0f1147368fb7bec39f25b43a)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:19.365825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:19.472952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a265e-d69e-4452-a40d-59e2ed27edaf None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:19.478502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cecfbacf0f1147368fb7bec39f25b43a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.483392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:19.484608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:19.485065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:19.487244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:19.491898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cecfbacf0f1147368fb7bec39f25b43a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.505719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:19.507668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:19.527567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1134/2255] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:16:19.533497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cecfbacf0f1147368fb7bec39f25b43a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.536315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cecfbacf0f1147368fb7bec39f25b43a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.550067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cecfbacf0f1147368fb7bec39f25b43a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:19.559451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1122/2256] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:19 2022] DELETE /identity/v3/users/cecfbacf0f1147368fb7bec39f25b43a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:20.327564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:20.330015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:20.353817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19418bd8-ab9b-4e1c-a6d4-f99ff717b083 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:20.374643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:20.376325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:20.491836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:20.493180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.493611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.493992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] PATH_INFO: `/v3/projects/71bd18157d964712866540452b6ca95c` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.496600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71bd18157d964712866540452b6ca95c)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:20.498933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:20.591890 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5eff2-a860-4669-b76c-5608195a42ab admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:20.606512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:20.607450 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.607753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.611586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] PATH_INFO: `/v3/projects/06739c5a7410476686556b679d82aa35` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.613980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=06739c5a7410476686556b679d82aa35)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:20.618887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:20.631274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71bd18157d964712866540452b6ca95c', 'request_id': 'req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.636155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:20.645251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71bd18157d964712866540452b6ca95c', 'request_id': 'req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.656707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71bd18157d964712866540452b6ca95c', 'request_id': 'req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.697333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] The token cache is being invalidate because project 71bd18157d964712866540452b6ca95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:20.697616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.700415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9a0d6379-4f7b-43bf-83bc-5dc5558c9a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.708556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1135/2257] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:20 2022] DELETE /identity/v3/projects/71bd18157d964712866540452b6ca95c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:20.708556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.708556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.708556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.709428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1136/2258] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:16:20.710203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:20.712005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:20.725296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:20.728856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06739c5a7410476686556b679d82aa35', 'request_id': 'req-47d412da-893d-4dbe-80d7-64e39ea34d8a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.748393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06739c5a7410476686556b679d82aa35', 'request_id': 'req-47d412da-893d-4dbe-80d7-64e39ea34d8a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.762370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06739c5a7410476686556b679d82aa35', 'request_id': 'req-47d412da-893d-4dbe-80d7-64e39ea34d8a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.788815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] The token cache is being invalidate because project 06739c5a7410476686556b679d82aa35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:20.789377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.790103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-47d412da-893d-4dbe-80d7-64e39ea34d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:20.799697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1123/2259] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:20 2022] DELETE /identity/v3/projects/06739c5a7410476686556b679d82aa35 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:20.801225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1bc4-229e-4eae-8eb1-0293c0b0f50f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.801524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1bc4-229e-4eae-8eb1-0293c0b0f50f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.801809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1bc4-229e-4eae-8eb1-0293c0b0f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.845666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-527b1bc4-229e-4eae-8eb1-0293c0b0f50f None None] MFA Rules not processed for user `fe214bb1d7bd4e6abd8256cab5ff579f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:20.846984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-527b1bc4-229e-4eae-8eb1-0293c0b0f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:20.852027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7da0989-d5a7-439a-8604-3c2c08411872 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:20.865334 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:20.866604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.866604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.866986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] PATH_INFO: `/v3/users/026355dbea3843e1ad42eed6757a3404` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.870035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026355dbea3843e1ad42eed6757a3404)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:20.872044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:20.961461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1124/2260] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:16:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:20.962521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:20.962802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:20.963085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:20.984055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:20.985242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.010883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1125/2261] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:21.011800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.013800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.026870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026355dbea3843e1ad42eed6757a3404'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.029559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9aa6fb-cc7f-42cc-b512-b26e0f97906c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.037303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.037990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.038273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.038592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.041652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026355dbea3843e1ad42eed6757a3404'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.049429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026355dbea3843e1ad42eed6757a3404'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.050690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026355dbea3843e1ad42eed6757a3404'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.051415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.051639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026355dbea3843e1ad42eed6757a3404'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.052676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c617c-6f8b-4da9-840a-a29613e5f281 service placement] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.066140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1126/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:21.067511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.067930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.068976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.074995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1137/2263] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:20 2022] DELETE /identity/v3/users/026355dbea3843e1ad42eed6757a3404 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:21.087604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:21.088832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.172730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1127/2264] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:21.177879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.189606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55f4e-e846-45c5-a6ba-b1d62691e206 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.194395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.195083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.195532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.196078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.196563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:21.196877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.198510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.216725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1138/2265] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:21.218270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.236738 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de7e376-9ada-4b93-8037-14567ae569c1 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.242011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.242394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.243211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.243211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.244080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.246942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.302469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1128/2266] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:21 2022] POST /identity/v3/projects => generated 387 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:21.305042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.322262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ce7c1-aa47-45b1-a7ba-044076ca6df8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.401239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.403913 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.424761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:21.461195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.476143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:21.476468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.477333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.478806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.480056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.482118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.530679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1139/2267] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:21 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:21.535407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.550478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8492d-5bd0-4ae4-93da-c985ebd0d594 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.562503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6e8be-492d-47cf-a6e0-9d8895260106 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.563609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.564397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.564765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.575461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.576725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.577021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.577307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] PATH_INFO: `/v3/users/2481bb023ba64600b4a926faaa034a59` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.581497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2481bb023ba64600b4a926faaa034a59)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.581998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.582297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.584439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.588842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.631571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1140/2268] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:21.632657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.655062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2bbb1-2554-40ad-a3fc-44f5c9fd4005 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.666389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.667294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.667723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.668142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.669213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.671288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.682105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2481bb023ba64600b4a926faaa034a59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.693811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2481bb023ba64600b4a926faaa034a59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.701384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1141/2269] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:21.710506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.718533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2481bb023ba64600b4a926faaa034a59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.719312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2481bb023ba64600b4a926faaa034a59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.733790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af59291-6304-4d4a-9f25-e662e45189f3 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:21.737074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2481bb023ba64600b4a926faaa034a59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:21.749626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1129/2270] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:21 2022] DELETE /identity/v3/users/2481bb023ba64600b4a926faaa034a59 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:21.751957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:21.753184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.753536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.753970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] PATH_INFO: `/v3/projects/32e18f77d19f450e8e11acdb34149b2f/users/2864f4d258df4577a59e0c24cc8ffe3a/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.784195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e18f77d19f450e8e11acdb34149b2f, user_id=2864f4d258df4577a59e0c24cc8ffe3a, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:21.787539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b805499-fa5e-4c37-86a0-ce555ea94e31 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:21.808964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:21.813108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.829562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1142/2271] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:21 2022] PUT /identity/v3/projects/32e18f77d19f450e8e11acdb34149b2f/users/2864f4d258df4577a59e0c24cc8ffe3a/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:21.842100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:21.843179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:21.843721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:21.903766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] MFA Rules not processed for user `2864f4d258df4577a59e0c24cc8ffe3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:21.905640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:21.985418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9fc55-7b99-4921-9c2d-384c428cf7e5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.046159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1143/2272] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:22.048291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.051552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.088462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:22.109633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.111209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.111817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.112239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] PATH_INFO: `/v3/projects/8c8c677a7b38452dac369e77de668006` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.114019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c8c677a7b38452dac369e77de668006)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.116896 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.180612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4c6f2-6993-4f2a-ae39-500d3644d882 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.199513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.200392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.200683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.200972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.201698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.216443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.234952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c8c677a7b38452dac369e77de668006', 'request_id': 'req-f9b35280-a3f1-4437-b0ea-4ddc25143e33'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.248423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c8c677a7b38452dac369e77de668006', 'request_id': 'req-f9b35280-a3f1-4437-b0ea-4ddc25143e33'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.265135 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c8c677a7b38452dac369e77de668006', 'request_id': 'req-f9b35280-a3f1-4437-b0ea-4ddc25143e33'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.283547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] The token cache is being invalidate because project 8c8c677a7b38452dac369e77de668006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:22.283940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.285409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.286947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1144/2273] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:22 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:22.287929 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.293270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1130/2274] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:21 2022] DELETE /identity/v3/projects/8c8c677a7b38452dac369e77de668006 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:22.293270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.295263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.295413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.314989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:22.407486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943d74b-fc3e-4c9f-b5af-de001585c1c5 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.427295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35280-a3f1-4437-b0ea-4ddc25143e33 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.428398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.429563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.429849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.430907 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] PATH_INFO: `/v3/projects/2aefe42fc2be4abaa3e9fb80fdc6006a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.432169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aefe42fc2be4abaa3e9fb80fdc6006a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.433855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.434701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.434824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.435274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.435564 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.436458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.438818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.466920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1131/2275] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:22 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:22.467856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.470910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.489986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97f8dc-af70-4839-a123-22b2e392c296 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.492375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aefe42fc2be4abaa3e9fb80fdc6006a', 'request_id': 'req-10b93416-25d3-476f-a531-f7002cad0aab'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.503184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aefe42fc2be4abaa3e9fb80fdc6006a', 'request_id': 'req-10b93416-25d3-476f-a531-f7002cad0aab'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.509518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aefe42fc2be4abaa3e9fb80fdc6006a', 'request_id': 'req-10b93416-25d3-476f-a531-f7002cad0aab'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.527471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] The token cache is being invalidate because project 2aefe42fc2be4abaa3e9fb80fdc6006a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:22.528292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.528699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.530340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1145/2276] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:22 2022] DELETE /identity/v3/projects/2aefe42fc2be4abaa3e9fb80fdc6006a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:22.533364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.533364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.588762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.590041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.590444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.590928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] PATH_INFO: `/v3/users/c7c99c50d8b54813a49a56032e34d93f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.592220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c99c50d8b54813a49a56032e34d93f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.594211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.608780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:22.609236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b93416-25d3-476f-a531-f7002cad0aab admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.614675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.615604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.615895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.616204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] PATH_INFO: `/v3/users/392b8ea4f0c6480d9eb65ce93ec3e719` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.617608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=392b8ea4f0c6480d9eb65ce93ec3e719)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.619781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.643647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c99c50d8b54813a49a56032e34d93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.651836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c99c50d8b54813a49a56032e34d93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.665448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c99c50d8b54813a49a56032e34d93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.666214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c99c50d8b54813a49a56032e34d93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.669648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:22.674783 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c99c50d8b54813a49a56032e34d93f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.676979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '392b8ea4f0c6480d9eb65ce93ec3e719'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.682200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1132/2277] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:22 2022] DELETE /identity/v3/users/c7c99c50d8b54813a49a56032e34d93f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:22.683070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.684852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.697434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '392b8ea4f0c6480d9eb65ce93ec3e719'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.706602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '392b8ea4f0c6480d9eb65ce93ec3e719'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.707392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '392b8ea4f0c6480d9eb65ce93ec3e719'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.708017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '392b8ea4f0c6480d9eb65ce93ec3e719'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.725972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1146/2278] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:22 2022] DELETE /identity/v3/users/392b8ea4f0c6480d9eb65ce93ec3e719 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:22.730200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.731662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.774082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7f1db-78b3-4688-86ff-f723ae381d5c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.835302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188fea2-d390-4a5f-8434-b83727efa8ea admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.845817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.847562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.848176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.848724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] PATH_INFO: `/v3/users/939a4c546e2a4b69a99b4135e2f9ada7` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.851990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=939a4c546e2a4b69a99b4135e2f9ada7)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.855184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.866674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.867461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.867733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.868525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.869651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.871905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.886795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1133/2279] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:22.888766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.891334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.912188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8363f-ec43-4962-a885-751978c4708b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:22.919798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:22.920725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:22.921015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:22.921292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] PATH_INFO: `/v3/projects/4040e5554ae9488bb9b3c866542462af/users/cd96b82cbe6c401a88123b7b04bc09d9/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:22.924310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939a4c546e2a4b69a99b4135e2f9ada7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.941247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939a4c546e2a4b69a99b4135e2f9ada7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.951065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4040e5554ae9488bb9b3c866542462af, user_id=cd96b82cbe6c401a88123b7b04bc09d9, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:22.951356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939a4c546e2a4b69a99b4135e2f9ada7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.952193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939a4c546e2a4b69a99b4135e2f9ada7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.954326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939a4c546e2a4b69a99b4135e2f9ada7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:22.956975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:22.976954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1147/2280] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:22 2022] DELETE /identity/v3/users/939a4c546e2a4b69a99b4135e2f9ada7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:22.978089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.980553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:22.991275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1134/2281] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:22 2022] PUT /identity/v3/projects/4040e5554ae9488bb9b3c866542462af/users/cd96b82cbe6c401a88123b7b04bc09d9/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:22.996261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:22.999422 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.033033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.052406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.103292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a26d8-580e-4c98-a4d8-256f74abce1d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.149306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ffd2d1-9d5a-4351-9dba-e8575d1d5a6d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.157634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.158482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.158774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.159052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.160247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.162207 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.182078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1135/2282] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:23.183061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.185227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.227308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451bb28-bdb3-4487-8126-404ff4b12048 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.233583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.234507 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.234792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.235073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] PATH_INFO: `/v3/users/117bbd1474aa416eab2de79761c4b28c` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.236472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=117bbd1474aa416eab2de79761c4b28c)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.237907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.239319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.244534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.246244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.248044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] PATH_INFO: `/v3/projects/6a37f11a6f794a518cfcfa600caf2445` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.251412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a37f11a6f794a518cfcfa600caf2445)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.256713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.312033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117bbd1474aa416eab2de79761c4b28c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.323984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a37f11a6f794a518cfcfa600caf2445', 'request_id': 'req-e97789e0-00b9-4a50-a2d8-3bc4413acb59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.325361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117bbd1474aa416eab2de79761c4b28c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.332217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a37f11a6f794a518cfcfa600caf2445', 'request_id': 'req-e97789e0-00b9-4a50-a2d8-3bc4413acb59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.333001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117bbd1474aa416eab2de79761c4b28c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.333867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117bbd1474aa416eab2de79761c4b28c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.334498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117bbd1474aa416eab2de79761c4b28c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.338471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a37f11a6f794a518cfcfa600caf2445', 'request_id': 'req-e97789e0-00b9-4a50-a2d8-3bc4413acb59'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.348576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1148/2283] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:22 2022] DELETE /identity/v3/users/117bbd1474aa416eab2de79761c4b28c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:23.349637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.351117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.353672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] The token cache is being invalidate because project 6a37f11a6f794a518cfcfa600caf2445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:23.353972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.354631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.356344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1136/2284] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:23 2022] DELETE /identity/v3/projects/6a37f11a6f794a518cfcfa600caf2445 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:23.360011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.360011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.418516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a6da97-4fa6-4d2b-9de4-b2f23ce2c7ba admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.423035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.423940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.424222 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.424314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789e0-00b9-4a50-a2d8-3bc4413acb59 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.424494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] PATH_INFO: `/v3/projects/900c14e3201840af90408b14e701e0d2` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.425691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=900c14e3201840af90408b14e701e0d2)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.427691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.429179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.430032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.430496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.430496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.430671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] PATH_INFO: `/v3/projects/4040e5554ae9488bb9b3c866542462af/users/cd96b82cbe6c401a88123b7b04bc09d9/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.432646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4040e5554ae9488bb9b3c866542462af, user_id=cd96b82cbe6c401a88123b7b04bc09d9, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.435749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.455587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1137/2285] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:23 2022] PUT /identity/v3/projects/4040e5554ae9488bb9b3c866542462af/users/cd96b82cbe6c401a88123b7b04bc09d9/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:23.455587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.455587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.480405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.490212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900c14e3201840af90408b14e701e0d2', 'request_id': 'req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.498449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900c14e3201840af90408b14e701e0d2', 'request_id': 'req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.514198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900c14e3201840af90408b14e701e0d2', 'request_id': 'req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.522001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5d1e-d0b5-424a-ac4b-30ae158af9f9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.526998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] The token cache is being invalidate because project 900c14e3201840af90408b14e701e0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:23.527298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.528123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0cf9c98-c7cf-404a-b488-6bbcf4da3e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.529800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1149/2286] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:23 2022] DELETE /identity/v3/projects/900c14e3201840af90408b14e701e0d2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:23.530931 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cb91-ede0-4ded-af19-4fa440f75f5f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.531256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cb91-ede0-4ded-af19-4fa440f75f5f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.531669 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98cb91-ede0-4ded-af19-4fa440f75f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.553688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cb98cb91-ede0-4ded-af19-4fa440f75f5f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.555597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cb98cb91-ede0-4ded-af19-4fa440f75f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.600961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.601947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.602336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.602897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] PATH_INFO: `/v3/projects/bd62340ba4d9428f87bfef02a660ec8c` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.604090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd62340ba4d9428f87bfef02a660ec8c)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.606186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.637050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1150/2287] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.639220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f119e-5b75-42de-ac82-c14a73f3d7ff None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.639791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f119e-5b75-42de-ac82-c14a73f3d7ff None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.640367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f119e-5b75-42de-ac82-c14a73f3d7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.662402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-880f119e-5b75-42de-ac82-c14a73f3d7ff None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.663505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-880f119e-5b75-42de-ac82-c14a73f3d7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.673833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd62340ba4d9428f87bfef02a660ec8c', 'request_id': 'req-a14d44e7-bb88-4426-a168-a5c9c5087f24'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.683941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd62340ba4d9428f87bfef02a660ec8c', 'request_id': 'req-a14d44e7-bb88-4426-a168-a5c9c5087f24'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.691485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd62340ba4d9428f87bfef02a660ec8c', 'request_id': 'req-a14d44e7-bb88-4426-a168-a5c9c5087f24'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.704331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] The token cache is being invalidate because project bd62340ba4d9428f87bfef02a660ec8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:23.704681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.705306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a14d44e7-bb88-4426-a168-a5c9c5087f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:23.706885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1138/2288] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:23 2022] DELETE /identity/v3/projects/bd62340ba4d9428f87bfef02a660ec8c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:23.708173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4dea0-9d45-49fb-970e-ea5cdd764cd5 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.708458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4dea0-9d45-49fb-970e-ea5cdd764cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.708735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4dea0-9d45-49fb-970e-ea5cdd764cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.737695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-15c4dea0-9d45-49fb-970e-ea5cdd764cd5 None None] MFA Rules not processed for user `cd96b82cbe6c401a88123b7b04bc09d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.738953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1151/2289] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.739327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-15c4dea0-9d45-49fb-970e-ea5cdd764cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.747969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.748566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.748941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.767754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.769505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.782565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1152/2290] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.784698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.786339 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.794090 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.826537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:23.832418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1139/2291] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.834085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2241-44a5-43b9-aeb4-eeae5c363033 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.834439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2241-44a5-43b9-aeb4-eeae5c363033 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.834860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c2241-44a5-43b9-aeb4-eeae5c363033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.862139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2b0c2241-44a5-43b9-aeb4-eeae5c363033 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.863400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2b0c2241-44a5-43b9-aeb4-eeae5c363033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.864048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21e738-1886-46ac-943b-efa31d889f58 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.867671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.868327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.868607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.868885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.870876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.878573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1140/2292] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.878573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92b73-5c84-47f9-8fba-6ab1644beed2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.878573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92b73-5c84-47f9-8fba-6ab1644beed2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.878573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa92b73-5c84-47f9-8fba-6ab1644beed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.895054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:23.896722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:23.896834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2aa92b73-5c84-47f9-8fba-6ab1644beed2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.897961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2aa92b73-5c84-47f9-8fba-6ab1644beed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.906255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1141/2293] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.907541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d73ad-c641-44e9-a999-7660482d7633 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.907820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d73ad-c641-44e9-a999-7660482d7633 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.908106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d73ad-c641-44e9-a999-7660482d7633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.909186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1153/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:23.910287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.911667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.927771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-938d73ad-c641-44e9-a999-7660482d7633 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.929190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-938d73ad-c641-44e9-a999-7660482d7633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.939261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1142/2295] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.940653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.940955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.941237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.965573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:23.966974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:23.976563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1143/2296] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:23.980772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:23.986440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ef9b5-26bb-49f5-90f6-57418c414eb5 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:23.991818 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:23.992910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:23.993446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:23.993866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:23.995423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227e0e5-22aa-4b89-b4d7-48069892eba5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.003600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.004639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.004933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.005218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.005449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.006271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:24.006589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.007524 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.009236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.020269 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1154/2297] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:24.021753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.025524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1144/2298] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:24.026632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.036386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7ab1-7e99-4b35-aae3-97becf51805d service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.043327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.044270 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.044769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.045126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.045939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:24.046313 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.049040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.053391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4862d-b711-48b5-aee2-b95375b9b304 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.059545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.060305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.060597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.060877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.061629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:24.061942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.063722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.074105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1155/2299] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:24.075037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.079682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1145/2300] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:24.080555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.082270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.090427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c248ce1-db7b-468f-a634-6db8b9beafed admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.096118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09b612-282d-4592-a4ad-3f5e2ba0b01c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.098634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.099542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.099836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.100125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.101546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.101820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.102838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.103165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.103451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] PATH_INFO: `/v3/users/2864f4d258df4577a59e0c24cc8ffe3a` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.103569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.104978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2864f4d258df4577a59e0c24cc8ffe3a)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.107271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.146309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1156/2301] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/projects => generated 393 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.146309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.200445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2864f4d258df4577a59e0c24cc8ffe3a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.220581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2864f4d258df4577a59e0c24cc8ffe3a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.247404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2864f4d258df4577a59e0c24cc8ffe3a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.249083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2864f4d258df4577a59e0c24cc8ffe3a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.274296 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2864f4d258df4577a59e0c24cc8ffe3a'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.276282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3ab1-6098-44af-b7c4-681667af9a08 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.287572 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1146/2302] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:24 2022] DELETE /identity/v3/users/2864f4d258df4577a59e0c24cc8ffe3a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:24.289153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.291548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.295013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.295743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.296029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.296540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.297469 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.301109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.346717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1157/2303] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/projects => generated 381 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.347685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.350934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.381015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:24.384712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b367ba-3bfc-4b85-9a0e-0042fc6c2f87 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.395907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:24.423808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4964d-d628-4d2e-b557-2d03a477908d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.428832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.429532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.429877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.430090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.430783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.432770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0822e6-af90-4442-820a-ec0007456272 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.446326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.447004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.447320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.447599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.448304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.451040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.453972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1158/2304] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.455295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.455577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.455854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.476036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:24.477253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.486426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1147/2305] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.486426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.486426 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.486776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1159/2306] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:24.486865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.488336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.500354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd873f-cd53-499b-ae7e-53c47e82dfe2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.553152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad355fac-b16e-4da9-9ee9-7db502b17a71 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.556457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.557142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.557429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.557707 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.557895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.558408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.558889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.559250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.559805 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] PATH_INFO: `/v3/projects/e31ab1240bb2431e9cae6b1588d409e2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.560567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.560841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31ab1240bb2431e9cae6b1588d409e2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.562762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.583609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1160/2307] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.584758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.598661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec47937-edb7-44fe-9765-bc231454e552 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.604471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.605148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.605432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.605714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.607251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.608901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.616619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31ab1240bb2431e9cae6b1588d409e2', 'request_id': 'req-5da84565-13d0-482b-925d-92a27f85152f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.622712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1161/2308] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:24.622712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.630091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31ab1240bb2431e9cae6b1588d409e2', 'request_id': 'req-5da84565-13d0-482b-925d-92a27f85152f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.631038 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efd3e1-1943-47c9-b596-51446af7364d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.637527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31ab1240bb2431e9cae6b1588d409e2', 'request_id': 'req-5da84565-13d0-482b-925d-92a27f85152f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.658041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] The token cache is being invalidate because project e31ab1240bb2431e9cae6b1588d409e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:24.658343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.659004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.661010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1148/2309] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:24 2022] DELETE /identity/v3/projects/e31ab1240bb2431e9cae6b1588d409e2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:24.663932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.665281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.699796 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.700869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.701238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.701610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.702576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.704625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:24.705143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.729047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1162/2310] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:24.730639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.734189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.735287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84565-13d0-482b-925d-92a27f85152f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.739014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.739789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.740141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.740485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.742445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.747866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b0c113-9ea0-469b-b0fe-603514a2f8d7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.753226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.754068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.754356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.754485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.755244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.756199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.756855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.757647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.767821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1149/2312] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:24.767973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1163/2312] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:24.768506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.768647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.772008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.772125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.780472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:24.785209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec708ec-1bba-4bcc-a9e7-340c67f8b22a service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.786631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8605947-f2bb-4cbd-845a-19809ba623a1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.790499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51872502-2626-4815-82b1-872731416c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.791310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872502-2626-4815-82b1-872731416c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.791597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872502-2626-4815-82b1-872731416c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.791874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51872502-2626-4815-82b1-872731416c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.792601 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51872502-2626-4815-82b1-872731416c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.793039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.794158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.794873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51872502-2626-4815-82b1-872731416c75 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.795012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.795337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] PATH_INFO: `/v3/users/2f736ec69e7d43359a166a5f91e3c24a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.796628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f736ec69e7d43359a166a5f91e3c24a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.798610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.806067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1150/2313] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:24.806658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51872502-2626-4815-82b1-872731416c75 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.808586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-51872502-2626-4815-82b1-872731416c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.826476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51872502-2626-4815-82b1-872731416c75 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.833818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.834409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.834891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.835319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.836142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.838375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.853773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1151/2314] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:24.854651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.868214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4f28ec-069d-47de-a594-ad9043e2e0c6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.874807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.875598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.876263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.876263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.877108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.878780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.883578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f736ec69e7d43359a166a5f91e3c24a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.894685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1152/2315] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:24.894685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.903253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f736ec69e7d43359a166a5f91e3c24a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.905653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90d9e1-e771-4f20-9ac9-65bcad661e68 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:24.911472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:24.912538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.912650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f736ec69e7d43359a166a5f91e3c24a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.912731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.912893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] PATH_INFO: `/v3/projects/74b53f46d15c4465b9005939114e67a3/users/b6854699a1dc475ab11a89bcaf2af85a/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:24.913376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f736ec69e7d43359a166a5f91e3c24a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.914082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f736ec69e7d43359a166a5f91e3c24a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:24.930832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1164/2316] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:24 2022] DELETE /identity/v3/users/2f736ec69e7d43359a166a5f91e3c24a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:24.931474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b53f46d15c4465b9005939114e67a3, user_id=b6854699a1dc475ab11a89bcaf2af85a, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:24.932845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:24.936194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640e9f5-dd7b-4f41-878c-2f049138e232 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:24.941708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:24.969002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1153/2317] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:24 2022] PUT /identity/v3/projects/74b53f46d15c4465b9005939114e67a3/users/b6854699a1dc475ab11a89bcaf2af85a/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:24.970685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:24.971138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:24.971286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.004524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] MFA Rules not processed for user `787ef24488804250a58038c7378623ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:25.006705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.051623 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f38dfbf-fc93-4685-bd93-0c99441236cb admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.100053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1154/2318] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:24 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:25.100910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.112209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.133787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.134503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.135143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.135143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.135820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.147578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7371fbe-d7d3-4e68-9bfb-84bc63de85aa admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.164961 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1165/2319] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:24 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:25.167065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.167547 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.167926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.188911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:25.190377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.198930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1166/2320] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:25.200245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.201728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.208438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665b2da-76f5-49e6-9bc4-7c81d8d3f2fb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.213334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.214215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ef750-17a1-401c-87ee-8b525cdae2c0 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.214353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.214553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.214831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] PATH_INFO: `/v3/projects/4a5e183a36084c9fb82f3a3f7d24402a/users/abfd09ef27184291a36b43b6664740c5/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.219266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.220253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.220632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.221004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] PATH_INFO: `/v3/projects/32e18f77d19f450e8e11acdb34149b2f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.222283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e18f77d19f450e8e11acdb34149b2f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.224361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.229938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5e183a36084c9fb82f3a3f7d24402a, user_id=abfd09ef27184291a36b43b6664740c5, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.231998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:25.233060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.253174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1155/2321] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:25 2022] PUT /identity/v3/projects/4a5e183a36084c9fb82f3a3f7d24402a/users/abfd09ef27184291a36b43b6664740c5/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:25.254273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.260362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:25.288323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e18f77d19f450e8e11acdb34149b2f', 'request_id': 'req-7ec7175a-543f-4627-9fb1-86d1783c206a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.297816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e18f77d19f450e8e11acdb34149b2f', 'request_id': 'req-7ec7175a-543f-4627-9fb1-86d1783c206a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.306027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e18f77d19f450e8e11acdb34149b2f', 'request_id': 'req-7ec7175a-543f-4627-9fb1-86d1783c206a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.321804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] The token cache is being invalidate because project 32e18f77d19f450e8e11acdb34149b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:25.322362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.323044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.324439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4fdc9-805e-43a6-a0e6-fb7c63c02024 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.327249 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1167/2322] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:25 2022] DELETE /identity/v3/projects/32e18f77d19f450e8e11acdb34149b2f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:25.328855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.330465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.393666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.394581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.394869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.395204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] PATH_INFO: `/v3/projects/4c3f9024be20455db41082b1138d35e3/users/69a85b8fb58948f78ba641cc45c2b671/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.397747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7175a-543f-4627-9fb1-86d1783c206a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.402879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.403643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.403986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.404324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.406904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.413102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c3f9024be20455db41082b1138d35e3, user_id=69a85b8fb58948f78ba641cc45c2b671, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.415976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d8792-7eb1-41ca-a438-d0d6f266fc8b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.422081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.424122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.432022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1156/2323] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:25 2022] PUT /identity/v3/projects/4c3f9024be20455db41082b1138d35e3/users/69a85b8fb58948f78ba641cc45c2b671/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:25.433233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4251-1e8d-4c4f-917f-86d0ac88fe45 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.433593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4251-1e8d-4c4f-917f-86d0ac88fe45 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.433990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4251-1e8d-4c4f-917f-86d0ac88fe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.435262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1168/2324] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:25.435948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.438135 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.451987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0288f36f-d704-4abb-85be-a7921f7c7dd7 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:25.459719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-05bf4251-1e8d-4c4f-917f-86d0ac88fe45 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:25.467114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-05bf4251-1e8d-4c4f-917f-86d0ac88fe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.528278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:25.529881 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.530595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.531576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] PATH_INFO: `/v3/users/cd96b82cbe6c401a88123b7b04bc09d9` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.533858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd96b82cbe6c401a88123b7b04bc09d9)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:25.540170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:25.548816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1157/2325] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:25.550936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:25.551263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:25.551831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:25.613454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] MFA Rules not processed for user `69a85b8fb58948f78ba641cc45c2b671`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:25.615640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.649283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:25.660415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:25.663795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd96b82cbe6c401a88123b7b04bc09d9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.691629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd96b82cbe6c401a88123b7b04bc09d9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.711078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd96b82cbe6c401a88123b7b04bc09d9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.714114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd96b82cbe6c401a88123b7b04bc09d9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.716206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd96b82cbe6c401a88123b7b04bc09d9'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:25.745850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1169/2326] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:25 2022] DELETE /identity/v3/users/cd96b82cbe6c401a88123b7b04bc09d9 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:25.747781 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.752738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:25.769598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1158/2327] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:25.770260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:25.775301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.001315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbded1-6506-4260-8f53-6fa9e0cf2ad5 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.010548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05a1fe-e632-40a4-bc6f-9341023b262f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.058318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.059927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.060337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.060751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] PATH_INFO: `/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.064081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a0dcb352f848548c9ca2fa2b4d721e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.068854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.131973 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.138665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.139239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.140609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.146976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.212616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a0dcb352f848548c9ca2fa2b4d721e', 'request_id': 'req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.218299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.223303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.227389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a0dcb352f848548c9ca2fa2b4d721e', 'request_id': 'req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.234969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a0dcb352f848548c9ca2fa2b4d721e', 'request_id': 'req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.248311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] The token cache is being invalidate because project 40a0dcb352f848548c9ca2fa2b4d721e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:26.248945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.250201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.252700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1170/2328] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:25 2022] DELETE /identity/v3/projects/40a0dcb352f848548c9ca2fa2b4d721e => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:26.254351 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.258631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.275021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1159/2329] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:26.276553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.293329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.319787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74184b-f059-4586-be00-d6f7958d17b7 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.323081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:26.329852 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c10d9c-e57e-43db-8f7e-4aa834cfe4c5 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.334868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.336006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.336587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.336936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.350240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.407928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.409719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.410236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.410512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] PATH_INFO: `/v3/projects/4040e5554ae9488bb9b3c866542462af` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.412188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4040e5554ae9488bb9b3c866542462af)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.415543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.430404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.432395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160ebb6-9e29-4733-abac-be28bd2fcdbb service glance] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.449014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:26.458428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1171/2330] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:26.462901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7691158-c1fe-44fd-be31-91d8043392ab None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.463310 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7691158-c1fe-44fd-be31-91d8043392ab None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.463622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7691158-c1fe-44fd-be31-91d8043392ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.494954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e7691158-c1fe-44fd-be31-91d8043392ab None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:26.496872 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e7691158-c1fe-44fd-be31-91d8043392ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.500812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4040e5554ae9488bb9b3c866542462af', 'request_id': 'req-a3cf4c59-1170-4555-8b22-53f119641151'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.518156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4040e5554ae9488bb9b3c866542462af', 'request_id': 'req-a3cf4c59-1170-4555-8b22-53f119641151'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.525563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4040e5554ae9488bb9b3c866542462af', 'request_id': 'req-a3cf4c59-1170-4555-8b22-53f119641151'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.562849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] The token cache is being invalidate because project 4040e5554ae9488bb9b3c866542462af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:26.563277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.593785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a3cf4c59-1170-4555-8b22-53f119641151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:26.596862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1160/2331] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:26 2022] DELETE /identity/v3/projects/4040e5554ae9488bb9b3c866542462af => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:26.602469 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e0ef-0b6d-4208-87fe-5eceba8e23fe None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.602900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e0ef-0b6d-4208-87fe-5eceba8e23fe None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.603397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241e0ef-0b6d-4208-87fe-5eceba8e23fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.630720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0241e0ef-0b6d-4208-87fe-5eceba8e23fe None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:26.632333 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0241e0ef-0b6d-4208-87fe-5eceba8e23fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.700571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1172/2332] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:26.709810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.710109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.710449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.748613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1161/2333] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:26.750481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.750838 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.751163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.763874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:26.766866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.777457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1173/2334] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:26.782117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.798138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367f2e6-4d36-437c-b8b3-f9aa39cf2b05 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.806979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.807556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.808097 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.808683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.809871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:26.810566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.812666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.830433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1174/2335] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:26.833568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:26.834810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:26.836061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.856170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d5782-1273-49c2-bdb0-2bd24b226547 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.863621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.863991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1162/2336] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:26.864543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.864915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.865046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.865307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.866884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.869281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.877078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172790-c671-426a-ac1f-c90bd4953fbe None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:26.889013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:26.891282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:26.891643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:26.891927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:26.893228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:26.893642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:26.897744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:26.930662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1175/2337] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:26.932009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.951990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1163/2338] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:26.953369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:26.980233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171960-c564-4ab8-86f8-1ffadefd8eb1 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.089192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa974b04-ffcd-4930-943f-682ef38694bc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.096656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.097619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.097982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.098320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.099202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.113035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.138430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.140205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.140501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.141883 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.143436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.147784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.159188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:27.200385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1176/2339] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/users => generated 285 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:27.202017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.218096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1164/2340] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:26 2022] POST /identity/v3/projects => generated 399 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:27.218379 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9c18e-410e-4112-807a-644de6ea5237 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.219834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.261103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:27.349518 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.351417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.351951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.352352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.356318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.360593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.369637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f652f45-562a-47c5-bcb9-b633ae9884f9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.384857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.385637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.385927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.386209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.387516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.391850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.395678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1177/2341] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:27.397171 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.436070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14df429-8e28-462a-9983-1e6ad193be1e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.453926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1165/2342] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:27 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:27.455792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.457749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.458420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.458828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.459510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.459959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.463360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.488095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd93f4-5a64-46c6-b69a-3b67dcff1791 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.496539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1178/2343] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:27.501587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.503677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.504573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.504862 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.505139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.507133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.511563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.525525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f11c-68de-42fe-80fa-7e2aa83ccd25 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.536763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1166/2344] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:27.537765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.542015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.545574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.548082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.548573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] PATH_INFO: `/v3/projects/7c849e14f39e4600a8e1264592a8775a/users/630c1acc5b00451884dee34094b3476c/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.550441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc448-944b-4cc9-9cae-0f5bf9e3d823 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.555577 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.556315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.556605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.557126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.558458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.560948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050f3c4-f3d0-448e-a2a4-0a33e6dcc515 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.588485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c849e14f39e4600a8e1264592a8775a, user_id=630c1acc5b00451884dee34094b3476c, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.595224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1167/2345] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:27.595224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c6ac-c0d8-439d-b1db-c8fd55d4b012 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.595224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c6ac-c0d8-439d-b1db-c8fd55d4b012 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.595224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1c6ac-c0d8-439d-b1db-c8fd55d4b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.596570 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.627622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1179/2346] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:27 2022] PUT /identity/v3/projects/7c849e14f39e4600a8e1264592a8775a/users/630c1acc5b00451884dee34094b3476c/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:27.629103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.650504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7ba1c6ac-c0d8-439d-b1db-c8fd55d4b012 None None] MFA Rules not processed for user `abfd09ef27184291a36b43b6664740c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:27.655086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7ba1c6ac-c0d8-439d-b1db-c8fd55d4b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:27.759602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385ef9-dba1-4bdb-8fa0-ef8ecd1c889d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:27.764831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:27.765672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.765957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.766237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] PATH_INFO: `/v3/projects/d42a0f2e6b134281bd5c4a0ab138ae26/users/6c35811d4e01474493b7d122dd80b88f/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.790618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1168/2347] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:27.792447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43a0-a760-4cae-87b6-2a2eb6b7732f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.792737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43a0-a760-4cae-87b6-2a2eb6b7732f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.793018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed43a0-a760-4cae-87b6-2a2eb6b7732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.812044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d42a0f2e6b134281bd5c4a0ab138ae26, user_id=6c35811d4e01474493b7d122dd80b88f, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:27.818953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:27.838874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-d3ed43a0-a760-4cae-87b6-2a2eb6b7732f None None] MFA Rules not processed for user `630c1acc5b00451884dee34094b3476c`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:27.840851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d3ed43a0-a760-4cae-87b6-2a2eb6b7732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:27.864551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1180/2348] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:27 2022] PUT /identity/v3/projects/d42a0f2e6b134281bd5c4a0ab138ae26/users/6c35811d4e01474493b7d122dd80b88f/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:27.865787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:27.876027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:27.932795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:27.975909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1169/2349] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:27.977815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:27.978270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:27.978704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:27.999840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1d214-99fc-4413-af56-6ed21fc88bdc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.007982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.011214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:28.011879 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.012500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.012992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.058770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] MFA Rules not processed for user `6c35811d4e01474493b7d122dd80b88f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:28.060983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:28.147360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.149000 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.166123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1181/2350] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:28.167073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.169019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:28.175030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1170/2351] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:28.179514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.301274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9b7e-172d-4ff0-ab5a-fc04269f00cd service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.320363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.321043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.321323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.321601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.354185 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.356606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.365004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdc05-d637-4b6b-ab2a-41bd43bef295 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.400679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.401000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.401228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.401511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.416601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1182/2352] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:28.419076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.426235 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.503163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1171/2353] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:28 2022] POST /identity/v3/projects => generated 399 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:28.506922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.530605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676fd011-fbff-4d4d-9e81-179b208e9d61 service glance] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.619912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.620857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.621149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.621502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.622585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.625111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.717319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1183/2354] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:28 2022] POST /identity/v3/users => generated 306 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:28.718693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.732290 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eaddf5-bfa2-44b3-93c3-ae5f47272d23 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.738714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.739795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.740163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.740427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.741211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.743024 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.779820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1172/2355] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:28.782242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.817858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70414e1e-76b6-45fd-b1ed-04d5ff6f0838 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:28.830578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:28.832537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:28.832900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:28.833286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] PATH_INFO: `/v3/projects/6e8ef1292675441e9863139a54199889/users/b4f7b385dcbd43dca6c47087840f9aa6/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:28.860700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8ef1292675441e9863139a54199889, user_id=b4f7b385dcbd43dca6c47087840f9aa6, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:28.865919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:28.870606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:28.899845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1184/2356] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:28 2022] PUT /identity/v3/projects/6e8ef1292675441e9863139a54199889/users/b4f7b385dcbd43dca6c47087840f9aa6/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:28.901599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:28.928000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55426f78-cea6-4485-9617-8b195784c521 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:29.032758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:29.033534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:29.034104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:29.034853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:29.036034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:29.038336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cde329-58df-4f7b-8339-ebf3302cfc6c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:29.049683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1173/2357] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:29.054369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:29.069047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48da1b-bcf4-43c7-837c-15971a96db08 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:29.074981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:29.076010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:29.076775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:29.077073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] PATH_INFO: `/v3/projects/6e8ef1292675441e9863139a54199889/users/b4f7b385dcbd43dca6c47087840f9aa6/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:29.079720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8ef1292675441e9863139a54199889, user_id=b4f7b385dcbd43dca6c47087840f9aa6, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:29.082588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:29.100444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1185/2358] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:29 2022] PUT /identity/v3/projects/6e8ef1292675441e9863139a54199889/users/b4f7b385dcbd43dca6c47087840f9aa6/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:29.113861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:29.114843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:29.114843 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:29.125964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:29.154655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] MFA Rules not processed for user `b4f7b385dcbd43dca6c47087840f9aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:29.156027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:29.271360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1174/2359] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:29.281301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:29.283556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:29.407603 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832acb61-8290-44ed-a01e-bde154646e4e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:29.412376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:29.413116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:29.413404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:29.413766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:29.427882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:29.429464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee9fc7-3475-408b-942e-0178d40e3aab service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:29.443613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1186/2360] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:29.568380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:29.582410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c293a8-2710-4a12-9228-1ea0b331b5d2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:29.592585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:29.594119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:29.594668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:29.597627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:29.706825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:29.709119 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662b36f1-83e5-4e39-b412-121f754beda2 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:29.735192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1175/2361] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:30.227906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529c98e-a303-49a0-9169-83bd3deed3dc None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:30.228602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529c98e-a303-49a0-9169-83bd3deed3dc None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:30.228602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529c98e-a303-49a0-9169-83bd3deed3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:30.266832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0529c98e-a303-49a0-9169-83bd3deed3dc None None] MFA Rules not processed for user `abfd09ef27184291a36b43b6664740c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:30.268315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0529c98e-a303-49a0-9169-83bd3deed3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:30.378681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1187/2362] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:32.921343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:32.921996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:32.922093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:32.989316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] MFA Rules not processed for user `b6854699a1dc475ab11a89bcaf2af85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:32.990763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:33.071484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1176/2363] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:33.088754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483d2e-02d3-4f4d-acd8-9e65bfa1f1ff None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:33.089331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483d2e-02d3-4f4d-acd8-9e65bfa1f1ff None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:33.089331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80483d2e-02d3-4f4d-acd8-9e65bfa1f1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:33.112848 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-80483d2e-02d3-4f4d-acd8-9e65bfa1f1ff None None] MFA Rules not processed for user `b6854699a1dc475ab11a89bcaf2af85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:33.114884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-80483d2e-02d3-4f4d-acd8-9e65bfa1f1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:33.126253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1188/2364] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:34.416599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:34.436640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:34.484665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092f064-ca31-4470-b4a0-60b1128e7528 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:34.490241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:34.491249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:34.491550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:34.491827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] PATH_INFO: `/v3/users/6c35811d4e01474493b7d122dd80b88f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:34.493206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c35811d4e01474493b7d122dd80b88f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:34.495218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:34.547120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c35811d4e01474493b7d122dd80b88f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:34.553894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c35811d4e01474493b7d122dd80b88f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:34.568605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c35811d4e01474493b7d122dd80b88f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:34.569503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c35811d4e01474493b7d122dd80b88f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:34.584724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c35811d4e01474493b7d122dd80b88f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:34.591451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1177/2365] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:34 2022] DELETE /identity/v3/users/6c35811d4e01474493b7d122dd80b88f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:34.596003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:34.596486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:34.596604 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:34.614784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:34.616111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:34.695276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1189/2366] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:34.705179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:34.706685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:34.772998 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736bf32-35ea-4f65-b92f-3b74aa9a8b76 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:34.777521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:34.778479 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:34.778762 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:34.779198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:34.789625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:34.791073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:34.801560 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1178/2367] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:35.115266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:35.117775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:35.142624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6698537-f30d-4746-9e58-e3e16667de23 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:35.177167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:35.178520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:35.178878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:35.179248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] PATH_INFO: `/v3/projects/d42a0f2e6b134281bd5c4a0ab138ae26` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:35.180720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d42a0f2e6b134281bd5c4a0ab138ae26)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:35.182976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:35.208568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:35.239622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42a0f2e6b134281bd5c4a0ab138ae26', 'request_id': 'req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.250338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42a0f2e6b134281bd5c4a0ab138ae26', 'request_id': 'req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.257755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42a0f2e6b134281bd5c4a0ab138ae26', 'request_id': 'req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.272330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] The token cache is being invalidate because project d42a0f2e6b134281bd5c4a0ab138ae26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:35.272556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.273241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.278776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1190/2368] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:35 2022] DELETE /identity/v3/projects/d42a0f2e6b134281bd5c4a0ab138ae26 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:35.279849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:35.290388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93f1bd-2baa-4f4f-8a7c-a13b35c32aba service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:35.335567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:35.357218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:35.358117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:35.358402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:35.358684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] PATH_INFO: `/v3/users/b4f7b385dcbd43dca6c47087840f9aa6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:35.359945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4f7b385dcbd43dca6c47087840f9aa6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:35.361901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:35.423666 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4f7b385dcbd43dca6c47087840f9aa6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.432685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4f7b385dcbd43dca6c47087840f9aa6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.449808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4f7b385dcbd43dca6c47087840f9aa6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.450743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4f7b385dcbd43dca6c47087840f9aa6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.464720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4cc535bd-f340-4128-89cc-db09b74138bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4f7b385dcbd43dca6c47087840f9aa6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:35.474634 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1179/2369] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:35 2022] DELETE /identity/v3/users/b4f7b385dcbd43dca6c47087840f9aa6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:35.827813 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:35.829294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:35.843146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4f3b-bec2-4613-b022-0e1cf78535a3 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:35.938087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:35.939197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:35.939652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:35.940109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] PATH_INFO: `/v3/projects/6e8ef1292675441e9863139a54199889` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:35.941509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8ef1292675441e9863139a54199889)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:35.943871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.018116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8ef1292675441e9863139a54199889', 'request_id': 'req-148c55ec-5376-45c8-b7bb-659d3366c46a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:36.042595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8ef1292675441e9863139a54199889', 'request_id': 'req-148c55ec-5376-45c8-b7bb-659d3366c46a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:36.053194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8ef1292675441e9863139a54199889', 'request_id': 'req-148c55ec-5376-45c8-b7bb-659d3366c46a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:36.071665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] The token cache is being invalidate because project 6e8ef1292675441e9863139a54199889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:36.071967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:36.072711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-148c55ec-5376-45c8-b7bb-659d3366c46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:36.074629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1191/2370] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:35 2022] DELETE /identity/v3/projects/6e8ef1292675441e9863139a54199889 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:36.080352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.080589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.081521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.116220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:36.118482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:36.192912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1180/2371] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:36.201892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992835e-704b-406b-a640-5358fa334336 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.202191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992835e-704b-406b-a640-5358fa334336 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.202477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992835e-704b-406b-a640-5358fa334336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.222987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d992835e-704b-406b-a640-5358fa334336 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:36.224242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d992835e-704b-406b-a640-5358fa334336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:36.232743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1192/2372] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:36.236952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.247211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97130e83-1a19-434e-a2e9-14de9d024083 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.251936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.252618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.252905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.253186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.253930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:36.254336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.255884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.267716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1181/2373] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:16:36.272242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992835e-704b-406b-a640-5358fa334336 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.282099 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992835e-704b-406b-a640-5358fa334336 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.287638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.288401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.288757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.289113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.289888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.291972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.327136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1193/2374] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:16:36.329882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.341804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9efaf9-d749-4a46-b4b2-179de42f6cfc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.393505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:36.408753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.409919 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.410271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.410639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.411459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.413565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.440441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1182/2375] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:36.441771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.453800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391f9af-bd51-4aab-bdb4-7c0d984c1b8c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.459113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.459835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.460119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.460393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.461141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.462800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.465655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:36.477383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1194/2376] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:36.480682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.491350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d1ed0-b46e-434c-bac1-5eb23948e195 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.496098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.496829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.497121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.497401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.498310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.499971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950a2d2-4191-4d50-9ec0-91ae223807e7 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.511077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1183/2377] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:36.515404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.526040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb999c-bff7-4e52-9ee4-460e3f39e90f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.531204 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.532117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.532405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.532686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] PATH_INFO: `/v3/projects/65784499809d4e388b1f0f4852ac56e1/users/f29c0a4866644b599aec2748e7ea2d42/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.549553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=65784499809d4e388b1f0f4852ac56e1, user_id=f29c0a4866644b599aec2748e7ea2d42, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.552489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.570475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1195/2378] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:36 2022] PUT /identity/v3/projects/65784499809d4e388b1f0f4852ac56e1/users/f29c0a4866644b599aec2748e7ea2d42/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:16:36.585197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.585642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.586091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.619546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] MFA Rules not processed for user `f29c0a4866644b599aec2748e7ea2d42`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:36.620804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:36.691130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1184/2379] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:36.696174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.767332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f6d2c-9dfe-4b6e-9c54-f9e48f4e6f8f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.772561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.773747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.774052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.774305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.775447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.777854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.815639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1196/2380] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:36.819429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.831354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aad460-e0f1-432d-9060-3286f6ab8b0b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.892568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.893262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.893548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.893828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.894537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.896553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.925118 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1185/2381] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:36 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:36.929044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.939866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be55c57-fc94-4278-b8f7-4c848b5c5578 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.944539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.945268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.945564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.945844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:36.946658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:36.948280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:36.960142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1197/2382] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:36.964376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:36.975530 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325340a-fc05-4392-9d9c-5754b1e7f6ee admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:36.981176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:36.982144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:36.982338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:36.982632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] PATH_INFO: `/v3/projects/6b4c83d15d694e34ab8a89ba8635e735/users/c51371fb324f43608565e4e5d340e3ee/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:37.001598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4c83d15d694e34ab8a89ba8635e735, user_id=c51371fb324f43608565e4e5d340e3ee, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:37.005073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:37.022066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1186/2383] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:36 2022] PUT /identity/v3/projects/6b4c83d15d694e34ab8a89ba8635e735/users/c51371fb324f43608565e4e5d340e3ee/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:37.026105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:37.036824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e9a5b-8302-43d0-98d7-54a50e5886ab admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:37.113878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:37.115175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:37.115578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:37.115858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:37.116756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:37.118683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea85c36-764e-433c-b351-a164879181f0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:37.130927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1198/2384] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:37.134780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:37.145672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a7b06-245b-48e5-84ee-c4ff258c825b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:37.150279 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:37.151211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:37.151511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:37.151781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] PATH_INFO: `/v3/projects/6b4c83d15d694e34ab8a89ba8635e735/users/c51371fb324f43608565e4e5d340e3ee/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:37.154552 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4c83d15d694e34ab8a89ba8635e735, user_id=c51371fb324f43608565e4e5d340e3ee, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:37.157354 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:37.168966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1187/2385] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:37 2022] PUT /identity/v3/projects/6b4c83d15d694e34ab8a89ba8635e735/users/c51371fb324f43608565e4e5d340e3ee/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:37.179384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:37.179676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:37.179975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:37.214084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] MFA Rules not processed for user `c51371fb324f43608565e4e5d340e3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:37.215479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:37.251966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:37.292007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1199/2386] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:37.303991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:37.306134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:37.316833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:37.378231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c127b3-0cf8-419b-8257-db93faebc3dc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:37.381925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:37.382600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:37.382886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:37.383194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:37.393131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:37.394583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeda1c0-4b44-46c1-8a84-5e319de5be02 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:37.404802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1188/2387] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:38.001615 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:38.069952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70edddd-761a-41da-8559-ff8232c852ea None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:38.075208 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:38.075998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:38.076281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:38.076610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] PATH_INFO: `/v3/users/f29c0a4866644b599aec2748e7ea2d42` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:38.077959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f29c0a4866644b599aec2748e7ea2d42)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:38.080048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:38.130306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f29c0a4866644b599aec2748e7ea2d42'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.143129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f29c0a4866644b599aec2748e7ea2d42'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.150596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f29c0a4866644b599aec2748e7ea2d42'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.151409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f29c0a4866644b599aec2748e7ea2d42'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.152101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f29c0a4866644b599aec2748e7ea2d42'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.168381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1200/2388] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:38 2022] DELETE /identity/v3/users/f29c0a4866644b599aec2748e7ea2d42 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:38.173481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:38.173933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:38.174108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:38.191742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:38.192955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:38.252576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1189/2389] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:38.262792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:38.264403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:38.321159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:38.326668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bfcc4f-ff96-4ec1-9454-acc8749a4673 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:38.333111 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:38.334003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:38.334690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:38.335350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:38.347252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:38.348730 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:38.359561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1201/2390] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:38.657705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:38.659754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:38.672975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d358dab-d5f7-490d-bf33-95205804fbf2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:38.678105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:38.679022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:38.679359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:38.679670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] PATH_INFO: `/v3/projects/65784499809d4e388b1f0f4852ac56e1` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:38.680906 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65784499809d4e388b1f0f4852ac56e1)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:38.682864 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:38.731993 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65784499809d4e388b1f0f4852ac56e1', 'request_id': 'req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.739932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65784499809d4e388b1f0f4852ac56e1', 'request_id': 'req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.745468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65784499809d4e388b1f0f4852ac56e1', 'request_id': 'req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.758019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] The token cache is being invalidate because project 65784499809d4e388b1f0f4852ac56e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:38.758233 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.758845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.760497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1190/2391] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:38 2022] DELETE /identity/v3/projects/65784499809d4e388b1f0f4852ac56e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:38.764749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:38.775124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d8dfa-0eb5-4480-a417-f11ad5e814aa service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:38.837965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:38.838839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:38.839131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:38.839434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] PATH_INFO: `/v3/users/c51371fb324f43608565e4e5d340e3ee` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:38.840686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51371fb324f43608565e4e5d340e3ee)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:38.842619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:38.899540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51371fb324f43608565e4e5d340e3ee'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.912240 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51371fb324f43608565e4e5d340e3ee'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.919210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51371fb324f43608565e4e5d340e3ee'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.919883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51371fb324f43608565e4e5d340e3ee'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.920477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9ac92502-7233-4f56-9ea7-51cf59f8fa09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51371fb324f43608565e4e5d340e3ee'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:38.934325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1202/2392] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:38 2022] DELETE /identity/v3/users/c51371fb324f43608565e4e5d340e3ee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:39.218767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.220448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:39.234111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be079a8-414a-4dc2-bc5a-7bd3bd76b348 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.269909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:39.298352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.299361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.299647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.299927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] PATH_INFO: `/v3/projects/6b4c83d15d694e34ab8a89ba8635e735` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.301170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4c83d15d694e34ab8a89ba8635e735)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.303208 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.353833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4c83d15d694e34ab8a89ba8635e735', 'request_id': 'req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:39.363599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4c83d15d694e34ab8a89ba8635e735', 'request_id': 'req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:39.370656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4c83d15d694e34ab8a89ba8635e735', 'request_id': 'req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:39.383172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] The token cache is being invalidate because project 6b4c83d15d694e34ab8a89ba8635e735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:39.383462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:39.384120 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1fc2d7aa-83e6-4132-b716-d3f0abe3a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:39.385707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1191/2393] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:39 2022] DELETE /identity/v3/projects/6b4c83d15d694e34ab8a89ba8635e735 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:39.391129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.391608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.391718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.412540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:39.413774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:39.477858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1203/2394] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:39.486720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.487064 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.487452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.517070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:39.518757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:39.529408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1192/2395] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:39.533832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.544639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ec877-434f-4c65-a8d8-db64cb0cfc8f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.550695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.551413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.551699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.551977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.552726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:39.553037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.554765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.564556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:39.573158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1204/2396] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:39.577994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.588057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e00d9c-4c1d-45a2-9f65-5e81e71581b0 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.593767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.594443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.594727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.595005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.595758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.597671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.631127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1193/2397] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:39 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:39.636170 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.647874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a3a3a-539f-469c-bc90-44413a37f24d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.705853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.706537 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.706816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.707130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.707824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.709774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.732988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1205/2398] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:39 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:39.737239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.747718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c75462-bdbe-4e36-8a5c-dd263cd06971 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.752965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.753633 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.753942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.754196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.754921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.756576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.768036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1194/2399] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:39.772331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.785726 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24355f49-8c83-49ed-9230-9bbf09ea1ec0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.791394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.792083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.792365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.792645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.793589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.795332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543a6a8-deb5-42b8-8e3c-3faf0d6235ef admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.805908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1206/2400] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:39.810361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:39.821153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a24f72-8087-4460-bf64-c2e284e2c249 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:39.826319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:39.827257 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.827541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.827816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] PATH_INFO: `/v3/projects/567f977fff304773bbf1a34ea2de3472/users/747ffa8ee5c546e5b0588fc5adf5ea70/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.843163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=567f977fff304773bbf1a34ea2de3472, user_id=747ffa8ee5c546e5b0588fc5adf5ea70, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:39.845858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:39.861492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1195/2401] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:39 2022] PUT /identity/v3/projects/567f977fff304773bbf1a34ea2de3472/users/747ffa8ee5c546e5b0588fc5adf5ea70/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:39.873698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:39.874370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:39.874758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:39.914389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] MFA Rules not processed for user `747ffa8ee5c546e5b0588fc5adf5ea70`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:39.915764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:39.986821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1207/2402] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:39.998334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:40.000107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.062199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f4f11-fc11-401f-88e7-1312537a4bcb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:40.066013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:40.066667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.066952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.067270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.076147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:40.077621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43647ad-7ba9-4e25-96f6-30fb05765d63 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:40.086554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1196/2403] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:40.106419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:40.176432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504f009-1902-4edf-80a7-3f1df719abd8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:40.182766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:40.183774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.184059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.184347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] PATH_INFO: `/v3/users/747ffa8ee5c546e5b0588fc5adf5ea70` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.185635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=747ffa8ee5c546e5b0588fc5adf5ea70)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:40.187619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:40.193443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:40.238487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747ffa8ee5c546e5b0588fc5adf5ea70'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.250252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747ffa8ee5c546e5b0588fc5adf5ea70'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.257150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747ffa8ee5c546e5b0588fc5adf5ea70'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.257972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747ffa8ee5c546e5b0588fc5adf5ea70'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.258579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747ffa8ee5c546e5b0588fc5adf5ea70'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.273274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1208/2404] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:16:40 2022] DELETE /identity/v3/users/747ffa8ee5c546e5b0588fc5adf5ea70 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:40.278330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.278808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.279649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.285640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:40.299069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:40.301195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.359085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1197/2405] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:40.370073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:40.372327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.440398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfa517-dd81-479c-9b67-62757b203be5 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:40.444136 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:40.444800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.445090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.445372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.454977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:40.456462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384cb2b-84c1-4aa6-9d8a-d0094f0e4860 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:40.466044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1209/2406] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:40.742880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:40.744408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.757699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a75019-8cc2-4579-b526-497fa690f071 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:40.762639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:40.763627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.763931 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.764216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] PATH_INFO: `/v3/projects/567f977fff304773bbf1a34ea2de3472` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.765731 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=567f977fff304773bbf1a34ea2de3472)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:40.767745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:40.817807 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567f977fff304773bbf1a34ea2de3472', 'request_id': 'req-30ffca99-cee5-47cc-aedd-aa966dec03d0'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.825742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567f977fff304773bbf1a34ea2de3472', 'request_id': 'req-30ffca99-cee5-47cc-aedd-aa966dec03d0'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.831356 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567f977fff304773bbf1a34ea2de3472', 'request_id': 'req-30ffca99-cee5-47cc-aedd-aa966dec03d0'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.843821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] The token cache is being invalidate because project 567f977fff304773bbf1a34ea2de3472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:16:40.844083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.844815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-30ffca99-cee5-47cc-aedd-aa966dec03d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:16:40.846442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1198/2407] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:16:40 2022] DELETE /identity/v3/projects/567f977fff304773bbf1a34ea2de3472 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:40.852413 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.852712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.852993 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.873727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:40.875132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.946189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1210/2408] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:40.954771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:40.955065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:40.955375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:40.976613 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:40.978211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:40.987710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1199/2409] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:40.992122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.002536 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0b8ea-dfbe-4165-88c1-70395c7e62fd None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.007946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.008688 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.009050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.009542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.010336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:16:41.010656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.012596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.024447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1211/2410] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:16:41.028975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.039382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a57db3-bff3-4afa-baa3-d772519518a6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.044845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.045521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.045811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.046096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.046804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.048907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.076261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1200/2411] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:16:41 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:41.081727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.092700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e226c-4bda-48d7-8315-5ba6ca3fc21d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.111533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:41.162145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.162771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.163054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.163363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.164084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.166042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.188423 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1212/2412] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:16:41 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:16:41.192785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.204266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359468-b932-4b6d-94e2-6c913b084d5f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.209005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.209663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.209945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.210218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.210928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.212567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.223437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1201/2413] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:16:41.227551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.237807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b4f64-94d7-4cba-bee9-5719498257d2 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.243411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.244106 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.244367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.244649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.245381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.247128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30f1f8-16f5-46c8-a703-909a62a3a332 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.257986 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1213/2414] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:16:41.263184 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.277294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5718-2b9a-4ef5-8a33-eb529f0ea5c6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.283694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.284968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.285255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.285534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] PATH_INFO: `/v3/projects/29ec13c3f4a04d688912b49eb43a02e6/users/211c1fc524bf4b82b8662fe02748c815/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.301589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ec13c3f4a04d688912b49eb43a02e6, user_id=211c1fc524bf4b82b8662fe02748c815, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.304270 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.319635 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1202/2415] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:16:41 2022] PUT /identity/v3/projects/29ec13c3f4a04d688912b49eb43a02e6/users/211c1fc524bf4b82b8662fe02748c815/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:16:41.330616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efc5d-3fa4-46b0-b5c0-bb92b76e3b62 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.330985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efc5d-3fa4-46b0-b5c0-bb92b76e3b62 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.331298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05efc5d-3fa4-46b0-b5c0-bb92b76e3b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.363040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c05efc5d-3fa4-46b0-b5c0-bb92b76e3b62 None None] MFA Rules not processed for user `211c1fc524bf4b82b8662fe02748c815`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:41.364292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c05efc5d-3fa4-46b0-b5c0-bb92b76e3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:41.433243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1214/2416] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:16:41.444284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:41.446072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:41.475551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:41.509371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e0586-8c8e-4deb-b72a-15aefe256cb0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:41.513914 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:41.514561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:41.514844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:41.515126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:41.523647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:41.525110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:41.534999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1203/2417] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:42.902259 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680bcd6-b90b-41b5-9050-12cc8d1b5140 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:42.902768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680bcd6-b90b-41b5-9050-12cc8d1b5140 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:42.902887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680bcd6-b90b-41b5-9050-12cc8d1b5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:42.921376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7680bcd6-b90b-41b5-9050-12cc8d1b5140 None None] MFA Rules not processed for user `211c1fc524bf4b82b8662fe02748c815`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:42.922580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7680bcd6-b90b-41b5-9050-12cc8d1b5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:42.929981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1215/2418] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:43.807395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:43.809092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:43.887461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff5162-3491-4be6-8de6-c2c1afc3c896 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:43.893386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:43.894048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:43.894335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:43.894629 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:43.896457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:43.957895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:43.959500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae742c0-72b4-4c09-82bb-5c54519c6606 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:43.970881 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1204/2419] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:16:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:16:45.293464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:45.294446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:45.294963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:45.314904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] MFA Rules not processed for user `b6854699a1dc475ab11a89bcaf2af85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:45.316281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:45.332914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1216/2420] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:16:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:53.965909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=72 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:55.309691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:16:59.036493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:59.037962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:59.038305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:59.060884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:16:59.062318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:59.137609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1205/2421] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:16:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:16:59.200032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:16:59.201535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:16:59.291473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d4176-d2ad-44dc-bc7f-22770325d9ba None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:16:59.298715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:16:59.299640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:16:59.300010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:16:59.300508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:16:59.310855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:16:59.313092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b336d31-1ac3-45a6-93e6-75a346e27887 service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:16:59.335256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1217/2422] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:16:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:09.123543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=25 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:09.325137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=32 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:11.020140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:11.021773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:11.153447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f112b9-d485-442d-9bb8-96ceedc1eef6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:11.159217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:11.160229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:11.160624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:11.161835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] PATH_INFO: `/v3/users/630c1acc5b00451884dee34094b3476c` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:11.162728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630c1acc5b00451884dee34094b3476c)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:11.164988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:11.254231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630c1acc5b00451884dee34094b3476c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:11.278967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630c1acc5b00451884dee34094b3476c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:11.313502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630c1acc5b00451884dee34094b3476c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:11.315565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630c1acc5b00451884dee34094b3476c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:11.342298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630c1acc5b00451884dee34094b3476c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:11.374865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1206/2423] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:17:11 2022] DELETE /identity/v3/users/630c1acc5b00451884dee34094b3476c => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:11.381733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:11.382706 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:11.383317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:11.421415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:11.424984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:11.628101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1218/2424] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:11.640438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:11.642811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:11.761033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657844d0-d857-430d-a151-d927ef8c88f4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:11.772327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:11.775005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:11.775860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:11.777327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:11.805183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:11.806758 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a664d93-8ee6-4b59-b3e0-2bd931aaf1b8 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:11.828800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1207/2425] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:11.889042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:11.890558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:11.905161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:11.905785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:11.906138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:11.922530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e5cc1-ee64-432f-9452-d9d2f799fcb5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:11.937088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:11.938019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:11.938295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:11.938576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] PATH_INFO: `/v3/users/69a85b8fb58948f78ba641cc45c2b671` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:11.939656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:11.940103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a85b8fb58948f78ba641cc45c2b671)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:11.941733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:11.942092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:12.003908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:12.098147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a85b8fb58948f78ba641cc45c2b671'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:12.142195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1208/2426] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:17:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:12.167113 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:12.169460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:12.276504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a85b8fb58948f78ba641cc45c2b671'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:12.343187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a85b8fb58948f78ba641cc45c2b671'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:12.343966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a85b8fb58948f78ba641cc45c2b671'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:12.349087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-93d8bb5b-d62d-4a37-8cb8-89bc09701559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a85b8fb58948f78ba641cc45c2b671'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:12.380925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1219/2427] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:17:11 2022] DELETE /identity/v3/users/69a85b8fb58948f78ba641cc45c2b671 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:12.387055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:12.388748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:12.390084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:12.449920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:12.453557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:12.455853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e123ba-3c5d-4ba2-bfb2-e4fedafbd91d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:12.545897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1220/2428] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:12.567429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:12.568905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:12.588332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:12.615502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:12.616190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:12.616492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:12.616760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:12.618818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:12.650121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7551d-9546-4d0f-b01b-db195d5d9de6 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:12.654160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:12.654943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:12.655345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:12.655918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:12.677790 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:12.680893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:12.694840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1221/2429] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:17:12.733132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:12.734770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:12.754512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1209/2430] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 04:17:13.081326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.084126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.107776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a927aa6-25d1-401d-b82c-2db87ab2158f service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:13.121072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:13.121984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.122271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.122554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] PATH_INFO: `/v3/projects/7c849e14f39e4600a8e1264592a8775a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.125372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c849e14f39e4600a8e1264592a8775a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:13.127456 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:13.267521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c849e14f39e4600a8e1264592a8775a', 'request_id': 'req-774f220f-c5ae-4549-b6df-49d4fb3e740a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.280909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c849e14f39e4600a8e1264592a8775a', 'request_id': 'req-774f220f-c5ae-4549-b6df-49d4fb3e740a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.297087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c849e14f39e4600a8e1264592a8775a', 'request_id': 'req-774f220f-c5ae-4549-b6df-49d4fb3e740a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.327428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] The token cache is being invalidate because project 7c849e14f39e4600a8e1264592a8775a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:17:13.327734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.332169 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-774f220f-c5ae-4549-b6df-49d4fb3e740a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.339129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.340374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1222/2431] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:17:13 2022] DELETE /identity/v3/projects/7c849e14f39e4600a8e1264592a8775a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:13.340620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.341954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2f33-a5a1-4f55-9606-b74f46ba56e2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.342254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2f33-a5a1-4f55-9606-b74f46ba56e2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.342676 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff2f33-a5a1-4f55-9606-b74f46ba56e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.374394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020d360-37d8-432d-a7b6-1ea8c9f846c9 service swift] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:13.401265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:13.408907 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b1ff2f33-a5a1-4f55-9606-b74f46ba56e2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:13.414807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b1ff2f33-a5a1-4f55-9606-b74f46ba56e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.464501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:13.465406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.465690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.466190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] PATH_INFO: `/v3/projects/4c3f9024be20455db41082b1138d35e3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.467830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c3f9024be20455db41082b1138d35e3)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:13.469420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:13.531077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c3f9024be20455db41082b1138d35e3', 'request_id': 'req-7548a3eb-e4da-42b0-b003-e30d5ffe6181'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.555897 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c3f9024be20455db41082b1138d35e3', 'request_id': 'req-7548a3eb-e4da-42b0-b003-e30d5ffe6181'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.568840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c3f9024be20455db41082b1138d35e3', 'request_id': 'req-7548a3eb-e4da-42b0-b003-e30d5ffe6181'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.622344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1223/2432] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:13.623904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.624193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.624747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.626504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] The token cache is being invalidate because project 4c3f9024be20455db41082b1138d35e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:17:13.626816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.630532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-7548a3eb-e4da-42b0-b003-e30d5ffe6181 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:13.634342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1210/2433] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:17:13 2022] DELETE /identity/v3/projects/4c3f9024be20455db41082b1138d35e3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:13.642853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.643180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.643476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.648425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:13.650024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.703172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:13.728756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:13.730043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.737220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1224/2434] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:13.741513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.752483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e67843-0a60-46d1-834b-4f1eed4abf76 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:13.757425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:13.758115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.759153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.759433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.760187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:17:13.760502 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:13.765398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf6450-213b-49a8-82fc-88d75474e4bb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:13.784965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1211/2435] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:13.795226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1225/2436] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 04:17:13.798304 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.798590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.798878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.809667 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.831754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133baeb-1ae1-4009-b664-a7eeae0351cb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:13.843546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:13.844242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:13.844531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:13.844811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:13.845865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:13.847277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:13.848707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:13.851126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:13.874743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1226/2437] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:13.876632 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.887131 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1212/2438] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:13.889716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:13.901089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b120b-76b8-4376-b8ce-1d4af7cf4528 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:13.998332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25eb780-34ad-47fd-b2cd-cfaf95a7642b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.009287 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.010212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.010506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.010781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.011526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.013745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.071055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.073182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.073572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.076098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1213/2439] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:13 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:14.077165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.079186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.080572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:17:14.082874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.088911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.097671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:14.107219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09387c2e-f361-42bb-9ce3-d10786261c1f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.114633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1227/2440] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:17:14.120816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.122820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.123567 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.123885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.124129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.130420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.132318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.146513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39cca0-e206-473c-81fb-5100024f1be1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.168851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1214/2441] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:14.170067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.189434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.190306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.190594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.190877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.191633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.191745 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a7a74-3101-4b9b-906e-82e5ff3b0132 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.193650 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.201702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.202646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.203128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.203876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.204835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.206622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.231865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1215/2442] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:17:14.233406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.248473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e2277-b12b-4206-9a7f-9a13d81df46a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.255664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.257115 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.257583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.258066 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] PATH_INFO: `/v3/projects/62d0dd3cae1448049dc95fb347fd1a80/users/ebfcd2137bf94726a25fea389ef24430/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.273975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1228/2443] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:14 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:14.276439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.296541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d0dd3cae1448049dc95fb347fd1a80, user_id=ebfcd2137bf94726a25fea389ef24430, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.301092 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedc6c4f-b234-4109-bd6f-60a00adef397 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.326943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1216/2444] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:14 2022] PUT /identity/v3/projects/62d0dd3cae1448049dc95fb347fd1a80/users/ebfcd2137bf94726a25fea389ef24430/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:14.333640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.334485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.334965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.397228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:14.415274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:14.416800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db14c3-0f84-49e6-90e2-2f94f8e7502e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.503847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:14.521873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1217/2445] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:14.534767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.539856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:14.547824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.548611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.548905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.549985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.551065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.553057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.621005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1229/2446] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:14 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:17:14.624021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.643290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa2401-ad77-4634-b07e-c136092f267b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.670784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.671575 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.671857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.672141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.672880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.675025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.694596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373d438-e600-4b90-b868-267a8f52dd82 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.705919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1230/2447] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:14.709237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.714536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.715565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.717376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.721589 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.733432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7743d4-8663-4c15-9c89-ae7162e0bb80 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.740082 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.742356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.742837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.743446 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.744647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.752338 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.762934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.764360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23b17d-3716-417c-b5a4-fd6fbeff9f03 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.786498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1231/2448] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:17:14.788039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:14.791698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1218/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:14.801125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab600cde-b836-4c17-b26f-de11264f516e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:14.807182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:14.809472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.809472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.809472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] PATH_INFO: `/v3/projects/a3238efa38c046c8bf2c8f863a99f08a/users/6567e226a8574eed8572e7c543e4c06d/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.829321 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3238efa38c046c8bf2c8f863a99f08a, user_id=6567e226a8574eed8572e7c543e4c06d, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:14.832435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:14.865138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1232/2450] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:14 2022] PUT /identity/v3/projects/a3238efa38c046c8bf2c8f863a99f08a/users/6567e226a8574eed8572e7c543e4c06d/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:14.876539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:14.877499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:14.877818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:14.989851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] MFA Rules not processed for user `6567e226a8574eed8572e7c543e4c06d`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:14.991242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:15.224945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:15.272005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1219/2451] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:15.298216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:15.312949 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:15.432230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf996-2f21-4c08-950d-b75f71d2a90b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:15.442332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:15.443465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:15.444067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:15.444573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:15.463366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:15.465201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd2fd2-899d-4bee-8be7-27a78fc592e4 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:15.492859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1233/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:18.725817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:18.998063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51400ff-3d54-4970-8be1-b6e2ed1d1825 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:19.044599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:19.045527 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:19.045808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:19.046079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] PATH_INFO: `/v3/users/6567e226a8574eed8572e7c543e4c06d` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:19.059647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6567e226a8574eed8572e7c543e4c06d)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:19.070780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:19.157885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:19.159193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:19.159610 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:19.204542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6567e226a8574eed8572e7c543e4c06d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:19.216502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6567e226a8574eed8572e7c543e4c06d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:19.217943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:19.219216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:19.237221 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6567e226a8574eed8572e7c543e4c06d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:19.238251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6567e226a8574eed8572e7c543e4c06d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:19.252929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-019f9b62-b421-4ed0-9811-61b63c60d8ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6567e226a8574eed8572e7c543e4c06d'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:19.264038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1220/2453] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:17:18 2022] DELETE /identity/v3/users/6567e226a8574eed8572e7c543e4c06d => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:19.268264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:19.269347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:19.269903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:19.306743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:19.308712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:19.455558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1221/2454] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:19.458831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1234/2455] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:17:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:17:19.477130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:19.478728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:19.493580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:19.494998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:19.499144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:19.574389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:19.616721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743126b-aa32-4aa0-950f-d71cb605cc0e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:19.684796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273066b9-f0fe-418c-80bc-0b741f9491b9 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:19.708902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:19.709599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:19.709887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:19.710174 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:19.763371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:19.764157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:19.764700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:19.765127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:19.791364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:19.794946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:19.829996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1235/2456] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:17:19.865125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:19.866768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf853dbc-b76b-4115-816f-f3d68c0de43d service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:19.888169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1222/2457] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:20.890062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:20.892125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:20.917549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ecdf33-0f2a-4e32-91ab-afa01e0bc55a service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:20.930279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:20.935296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:20.935636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:20.936013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] PATH_INFO: `/v3/projects/a3238efa38c046c8bf2c8f863a99f08a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:20.942592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3238efa38c046c8bf2c8f863a99f08a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:20.946485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:21.069378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3238efa38c046c8bf2c8f863a99f08a', 'request_id': 'req-8572270f-eac8-4383-93fd-dcaffa6325b7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:21.086474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3238efa38c046c8bf2c8f863a99f08a', 'request_id': 'req-8572270f-eac8-4383-93fd-dcaffa6325b7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:21.098971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3238efa38c046c8bf2c8f863a99f08a', 'request_id': 'req-8572270f-eac8-4383-93fd-dcaffa6325b7'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:21.120954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] The token cache is being invalidate because project a3238efa38c046c8bf2c8f863a99f08a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:17:21.121426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:21.122461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8572270f-eac8-4383-93fd-dcaffa6325b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:21.124714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1236/2458] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:17:20 2022] DELETE /identity/v3/projects/a3238efa38c046c8bf2c8f863a99f08a => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:21.130039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.131059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.131415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.165991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:21.167427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:21.353855 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1223/2459] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:21.361799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.362526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.363016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.396964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:21.398405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:21.415221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1237/2460] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:21.419255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:21.441311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa1a5-78e1-43c0-ac49-fe7a8906c5ed None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:21.471950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:21.471950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.471950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.471950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.472575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:17:21.478294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:21.481551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:21.515928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1224/2461] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:17:21.521901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:21.534034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2a398-5ba7-4cf7-afff-6df2f64fca1b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:21.548025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:21.549195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.549594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.552323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.553189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:21.555288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:21.630398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1238/2462] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:21 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:21.634353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:21.647984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04318990-aa31-43fc-83d9-b7720bfca016 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:21.755122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:21.814232 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:21.815173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.815457 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.815751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.816564 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:21.818614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:21.893345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1225/2463] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:21 2022] POST /identity/v3/users => generated 283 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:17:21.896162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:21.916710 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a819f29-4b4d-4e7c-a009-8937e6e44438 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:21.933384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:21.935620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:21.936658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:21.937160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:21.940968 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:21.945481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:21.982297 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1239/2464] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:21.986288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:22.001971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285671e2-0264-4d42-b9b3-1da9a685446e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:22.017045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:22.017760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:22.018050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:22.018329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:22.019074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:22.020948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc54c97-679c-4d2b-9212-de9f51a731db admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:22.038980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1226/2465] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:22.042587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:22.070468 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf157d9b-a0fa-45a4-aabc-ec607101241f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:22.077501 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:22.079560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:22.080145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:22.080667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] PATH_INFO: `/v3/projects/e277bba43c6a42218baa2f57f67298fc/users/058514d6c0594cf5887389c1838d3758/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:22.103448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e277bba43c6a42218baa2f57f67298fc, user_id=058514d6c0594cf5887389c1838d3758, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:22.107167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:22.147997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1240/2466] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:22 2022] PUT /identity/v3/projects/e277bba43c6a42218baa2f57f67298fc/users/058514d6c0594cf5887389c1838d3758/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:22.165781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490bc0-0f73-473b-8df0-209db4794ad1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:22.166327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490bc0-0f73-473b-8df0-209db4794ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:22.166435 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490bc0-0f73-473b-8df0-209db4794ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:22.215305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-78490bc0-0f73-473b-8df0-209db4794ad1 None None] MFA Rules not processed for user `058514d6c0594cf5887389c1838d3758`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:22.217339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-78490bc0-0f73-473b-8df0-209db4794ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:22.370911 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1227/2467] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:17:22.386043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:22.387950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:22.408857 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:22.547078 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a86ce5-f145-4645-b81b-7a554fde6b8a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:22.559043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:22.559864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:22.560213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:22.560563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:22.586250 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:22.588053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:22.600254 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1241/2468] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:24.530878 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:24.532054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:24.532595 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:24.575379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] MFA Rules not processed for user `ebfcd2137bf94726a25fea389ef24430`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:24.577334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:24.704702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1228/2469] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:24.717592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:24.719081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:24.900333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4321a6e-c829-4e40-95fa-040f0ce3655c service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:24.906764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:24.907558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:24.908407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:24.908859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:24.909583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:24.912378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:24.975137 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1242/2470] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:24 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:24.979988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:25.000325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfec66-2ea9-4aa4-bfab-369f9adb83b0 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:25.082275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:25.128472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:25.129449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:25.129842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:25.130214 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:25.131038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:25.133255 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:25.171082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1229/2471] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:24 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:25.172476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:25.185206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac52c6-f6fc-42bf-800d-678af7a4e946 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:25.190408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:25.191181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:25.191426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:25.191708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:25.192457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:25.194093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:25.216256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1243/2472] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:25.220200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:25.240239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916b8bd-da5b-4fcd-831c-cc9c8cbaef91 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:25.246537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:25.250413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:25.250538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:25.250744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] PATH_INFO: `/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9/users/dc0a9730c0774266a46d297498f7d496/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:25.295335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f161ea30f4f4748a753e6fe6f28efe9, user_id=dc0a9730c0774266a46d297498f7d496, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:25.307946 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:25.346997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1230/2473] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:25 2022] PUT /identity/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9/users/dc0a9730c0774266a46d297498f7d496/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:25.350855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:25.367224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb84e3-f326-4b9b-bafd-83c671c83ce8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:25.545187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:25.545187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:25.545829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:25.545829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:25.546605 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:25.550849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a821ae-6136-421c-a6b0-0987a0cbf526 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:25.586265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1244/2474] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:25.586506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:25.601476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1972c0f-7a18-4a55-8fc6-f2fa3714abfa admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:25.611984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:25.614106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:25.614774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:25.615404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] PATH_INFO: `/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9/users/dc0a9730c0774266a46d297498f7d496/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:25.617689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f161ea30f4f4748a753e6fe6f28efe9, user_id=dc0a9730c0774266a46d297498f7d496, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:25.621641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:25.642886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1231/2475] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:25 2022] PUT /identity/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9/users/dc0a9730c0774266a46d297498f7d496/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:30.874691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936f9e-4da5-41e7-b653-a62127181ff8 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:30.875368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936f9e-4da5-41e7-b653-a62127181ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:30.875368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4936f9e-4da5-41e7-b653-a62127181ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:30.910022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f4936f9e-4da5-41e7-b653-a62127181ff8 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:30.913785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f4936f9e-4da5-41e7-b653-a62127181ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:30.958951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:31.025567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1245/2476] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:17:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:31.042345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:31.044587 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:31.285639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea1592-cab6-48d9-bf5b-6e8d0175dccb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:31.299865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:31.300857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:31.301251 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:31.301641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:31.330107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:31.332010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:31.348869 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1232/2477] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:31.672937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d20dd-cd02-45fc-a28f-a9cf4de55c20 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:31.673316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d20dd-cd02-45fc-a28f-a9cf4de55c20 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:31.673607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d20dd-cd02-45fc-a28f-a9cf4de55c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:31.694946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b40d20dd-cd02-45fc-a28f-a9cf4de55c20 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:31.696261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b40d20dd-cd02-45fc-a28f-a9cf4de55c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:31.704393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1246/2478] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:17:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:31.725499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:31.747755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6111-43be-48b8-893d-8bef0b0ee7a8 service swift] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:31.753910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:31.754583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:31.754871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:31.755227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:31.766162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:31.768185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:31.778403 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1233/2479] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:32.091340 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0e0b-272e-4be8-896b-a011ceb65d86 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:32.091897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0e0b-272e-4be8-896b-a011ceb65d86 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:32.091897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a0e0b-272e-4be8-896b-a011ceb65d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:32.110746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f56a0e0b-272e-4be8-896b-a011ceb65d86 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:32.112323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f56a0e0b-272e-4be8-896b-a011ceb65d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:32.124630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1247/2480] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:32.134341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:32.146532 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21e43-199f-4bad-9857-866aa8c04961 service swift] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:32.152755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:32.153423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:32.153710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:32.153990 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:32.166128 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:32.167784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:32.181967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1234/2481] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:32.648890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29f28-983e-4c34-9d3a-3ad55070c03c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:32.649440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29f28-983e-4c34-9d3a-3ad55070c03c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:32.649974 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b29f28-983e-4c34-9d3a-3ad55070c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:32.677184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f8b29f28-983e-4c34-9d3a-3ad55070c03c None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:32.678620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f8b29f28-983e-4c34-9d3a-3ad55070c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:32.686937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1248/2482] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:32.700395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:32.711934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b33099-fa33-4aec-b888-5cecfd8eb71a service swift] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:32.720831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:32.721579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:32.722089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:32.722610 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:32.733155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:32.735146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:32.744520 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1235/2483] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:17:33.925360 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb354-a2df-445f-ac6d-cde9c9485d52 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:33.925842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb354-a2df-445f-ac6d-cde9c9485d52 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:33.926048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eeb354-a2df-445f-ac6d-cde9c9485d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:33.958761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-41eeb354-a2df-445f-ac6d-cde9c9485d52 None None] MFA Rules not processed for user `dc0a9730c0774266a46d297498f7d496`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:33.960089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-41eeb354-a2df-445f-ac6d-cde9c9485d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:34.042659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1249/2484] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:34.058632 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:34.060070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:34.102483 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:34.134840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da0d6c-e381-4f94-9ae7-bf5eb8b4127e service swift] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:34.140516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:34.141179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:34.141558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:34.142001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:34.199321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:34.201016 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:34.210815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1236/2485] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:35.466016 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ecc9-e6fa-45af-a991-b7d38ca17806 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:35.466546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ecc9-e6fa-45af-a991-b7d38ca17806 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:35.466652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ecc9-e6fa-45af-a991-b7d38ca17806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:35.492619 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5862ecc9-e6fa-45af-a991-b7d38ca17806 None None] MFA Rules not processed for user `ebfcd2137bf94726a25fea389ef24430`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:35.493853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5862ecc9-e6fa-45af-a991-b7d38ca17806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:35.501258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1250/2486] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:36.195573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:36.197616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:36.338368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8ad68-fea8-4c59-96e9-2ca3a57b0eca service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:36.349859 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:36.353087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:36.353978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:36.354865 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] PATH_INFO: `/v3/users/211c1fc524bf4b82b8662fe02748c815` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:36.358460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=211c1fc524bf4b82b8662fe02748c815)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:36.363695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:36.473695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '211c1fc524bf4b82b8662fe02748c815'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:36.512837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '211c1fc524bf4b82b8662fe02748c815'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:36.537636 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '211c1fc524bf4b82b8662fe02748c815'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:36.548797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '211c1fc524bf4b82b8662fe02748c815'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:36.553414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:36.569691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '211c1fc524bf4b82b8662fe02748c815'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:36.598729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1237/2487] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:17:36 2022] DELETE /identity/v3/users/211c1fc524bf4b82b8662fe02748c815 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:36.603020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:36.603894 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:36.605569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:36.634277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:36.635812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:36.733253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1251/2488] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:17:36.755619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:36.760337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:36.847847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21cc15-7cf7-4a5f-958b-f0d3e95db3dd admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:36.852097 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:36.852787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:36.853077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:36.853357 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:36.863619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:36.865095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fde0ed-16b1-40af-8023-6d86062a0cc4 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:36.876681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1238/2489] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:37.230642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:37.232239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:37.252963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4dd41-e72a-4b1b-b102-936e18f51dc2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:37.269664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:37.270957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.271296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.271579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] PATH_INFO: `/v3/projects/29ec13c3f4a04d688912b49eb43a02e6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.286322 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ec13c3f4a04d688912b49eb43a02e6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:37.290083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:37.331298 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:37.370904 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ec13c3f4a04d688912b49eb43a02e6', 'request_id': 'req-7f81da74-9231-45e6-b98e-573e3f5ea6f5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.387729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ec13c3f4a04d688912b49eb43a02e6', 'request_id': 'req-7f81da74-9231-45e6-b98e-573e3f5ea6f5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.403934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ec13c3f4a04d688912b49eb43a02e6', 'request_id': 'req-7f81da74-9231-45e6-b98e-573e3f5ea6f5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.448916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] The token cache is being invalidate because project 29ec13c3f4a04d688912b49eb43a02e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:17:37.449223 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.449900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.455649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1252/2490] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:17:37 2022] DELETE /identity/v3/projects/29ec13c3f4a04d688912b49eb43a02e6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:17:37.460737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a340f-492b-4c1e-9afb-931d43adac3e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.461701 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a340f-492b-4c1e-9afb-931d43adac3e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.462141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a340f-492b-4c1e-9afb-931d43adac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.492926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6e5a340f-492b-4c1e-9afb-931d43adac3e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:37.494178 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6e5a340f-492b-4c1e-9afb-931d43adac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:37.501654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:37.503636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:37.630303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1239/2491] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:37.637664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.638476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.639401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.653242 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f81da74-9231-45e6-b98e-573e3f5ea6f5 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:37.660500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:37.662014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.662451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.662928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] PATH_INFO: `/v3/users/787ef24488804250a58038c7378623ae` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.663525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:37.664634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=787ef24488804250a58038c7378623ae)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:37.664858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:37.668590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:37.675343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1240/2492] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:37.677725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:37.698003 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c59226-2e4a-47d5-9725-8eb43fc3c960 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:37.723656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:37.725310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.725884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.726468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.727523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:17:37.728012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:37.730058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:37.752292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1241/2493] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:17:37.764891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:37.779631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787ef24488804250a58038c7378623ae'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.791460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115927-a517-4a82-b739-99959ab09b24 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:37.808127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:37.809937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:37.810825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:37.811728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:37.813028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:37.816721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:37.822218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787ef24488804250a58038c7378623ae'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.842594 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787ef24488804250a58038c7378623ae'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.843574 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787ef24488804250a58038c7378623ae'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.844337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787ef24488804250a58038c7378623ae'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:37.852298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1242/2494] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:37 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:37.856588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:37.863071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1253/2495] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:17:37 2022] DELETE /identity/v3/users/787ef24488804250a58038c7378623ae => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:37.874699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:37.989460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:37.996320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe42f3-3eb8-4bea-b94f-a6971c64f80a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.073957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.074669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.074996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.075509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.076155 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.078145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.095732 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:38.097624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:38.111293 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1243/2496] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:37 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:38.114046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:38.117722 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69de4-d76a-4472-a33a-6b00dc42fecd admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.123560 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.124753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.125044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.125327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851669d-cb56-46d8-b2c2-3945474dcb8f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.125429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] PATH_INFO: `/v3/projects/94100e07e4d34a679ae537cb87de4dba` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.126966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94100e07e4d34a679ae537cb87de4dba)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.128802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.130592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.131382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.131674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.132134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.132961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.134671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.146078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1244/2497] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:38.150010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:38.160695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:38.161181 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5e601-0931-42f8-8aff-823c745871a0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.166710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.167432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.167741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.168014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.168749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.170526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.182474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1245/2498] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:17:38.186718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:38.190957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94100e07e4d34a679ae537cb87de4dba', 'request_id': 'req-b759d27d-e41f-48ce-955a-0475460d0516'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:38.197706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae084c1-d59d-4cb5-87a6-dd4c837404fd admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.199364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94100e07e4d34a679ae537cb87de4dba', 'request_id': 'req-b759d27d-e41f-48ce-955a-0475460d0516'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:38.205812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94100e07e4d34a679ae537cb87de4dba', 'request_id': 'req-b759d27d-e41f-48ce-955a-0475460d0516'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:38.223461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] The token cache is being invalidate because project 94100e07e4d34a679ae537cb87de4dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:17:38.224071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:38.225022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b759d27d-e41f-48ce-955a-0475460d0516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:17:38.227854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1254/2499] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:17:38 2022] DELETE /identity/v3/projects/94100e07e4d34a679ae537cb87de4dba => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:38.288441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.289382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.289663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.289940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] PATH_INFO: `/v3/projects/c2fe0f4b2519494ea784cea098da6be3/users/e269c6a656ff4721911ae25657eeedca/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.306389 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fe0f4b2519494ea784cea098da6be3, user_id=e269c6a656ff4721911ae25657eeedca, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.309176 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.324730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1246/2500] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:38 2022] PUT /identity/v3/projects/c2fe0f4b2519494ea784cea098da6be3/users/e269c6a656ff4721911ae25657eeedca/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:38.330075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4978a-3217-48bd-a8e3-f8b2e49ad950 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.330384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4978a-3217-48bd-a8e3-f8b2e49ad950 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.330627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4978a-3217-48bd-a8e3-f8b2e49ad950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.350003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cad4978a-3217-48bd-a8e3-f8b2e49ad950 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:38.351159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cad4978a-3217-48bd-a8e3-f8b2e49ad950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:38.431928 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1255/2501] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:38.449323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:38.451321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:38.513927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ba16b-5479-4503-9069-53709319c7b9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:38.518328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:38.518991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:38.519311 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:38.519591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:38.528179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:38.530729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:38.533905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70752f00-9de5-4bb9-bf33-7fb04fe5f197 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:38.546365 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1247/2502] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:17:39.725173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:39.725646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:39.725808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:39.727215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1256/2503] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:17:39.734203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953144d2-60b9-43e2-aad6-e0801288eb8a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:39.734736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953144d2-60b9-43e2-aad6-e0801288eb8a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:39.734736 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953144d2-60b9-43e2-aad6-e0801288eb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:39.764223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-953144d2-60b9-43e2-aad6-e0801288eb8a None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:39.775565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-953144d2-60b9-43e2-aad6-e0801288eb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:39.935319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1248/2504] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:17:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:39.950140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:39.951978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:40.049684 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de6654-68ca-415a-9e5b-c5441f974ea8 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:40.053913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:40.054653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.054936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.055320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.090488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:40.092640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c0690-929b-4d3f-ae2e-1bfc673b333d service glance] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:40.106694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1257/2505] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:40.222096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef084-f820-49d5-9c24-7f5c538a012b None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.222400 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef084-f820-49d5-9c24-7f5c538a012b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.222686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef084-f820-49d5-9c24-7f5c538a012b None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.224559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1249/2506] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:17:40.230193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94155d4-3dc6-4198-8cb5-c1675b4567b4 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.231325 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94155d4-3dc6-4198-8cb5-c1675b4567b4 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.231736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94155d4-3dc6-4198-8cb5-c1675b4567b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.264305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b94155d4-3dc6-4198-8cb5-c1675b4567b4 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:40.265631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b94155d4-3dc6-4198-8cb5-c1675b4567b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:40.328417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1258/2507] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:17:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:40.342887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef084-f820-49d5-9c24-7f5c538a012b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:40.368795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef084-f820-49d5-9c24-7f5c538a012b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:40.378046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:40.378873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.379511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.379885 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.405968 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:40.410814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c454f5-6637-4082-b86b-c381421c4706 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:40.457582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1250/2508] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:17:40.564787 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b382-36c1-4056-93b8-2efbd0d8f2d1 None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.565091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b382-36c1-4056-93b8-2efbd0d8f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.565383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b382-36c1-4056-93b8-2efbd0d8f2d1 None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.569393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1259/2509] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:17:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:17:40.591699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a2ad-0fe1-4674-ad25-5534a2362037 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.592234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a2ad-0fe1-4674-ad25-5534a2362037 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.592721 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291a2ad-0fe1-4674-ad25-5534a2362037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.635774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a291a2ad-0fe1-4674-ad25-5534a2362037 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:40.639794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a291a2ad-0fe1-4674-ad25-5534a2362037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:40.744550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1251/2510] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:17:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:40.756511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42b382-36c1-4056-93b8-2efbd0d8f2d1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:40.798503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42b382-36c1-4056-93b8-2efbd0d8f2d1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:40.815380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:40.818335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:40.820174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:40.820539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:40.857725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:40.859662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:40.883768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1260/2511] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:43.771472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:43.771943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:43.772035 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:43.854751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] MFA Rules not processed for user `e269c6a656ff4721911ae25657eeedca`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:43.857376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:43.943145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1252/2512] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:43.947748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:43.949217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:43.977300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:44.006092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b3bce-0687-4e95-801e-0a259fffdf19 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.010864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.011583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.011867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.012145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.012843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.014752 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.054383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1261/2513] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:17:43 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:44.055617 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:44.067349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857238dc-8055-4fc5-b536-438a4166dd2c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.080927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:44.133757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.134473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.134943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.135254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.135976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.137940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.163924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1253/2514] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:17:44 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:17:44.167922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:44.179764 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63770da8-e81c-46f8-b1cc-9bfdbfbcc480 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.185035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.185737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.186030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.186307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.187050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.189164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.216426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1262/2515] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:44.217307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:44.230444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd5780-b174-4d3f-af01-37af03cc8c83 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.235256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.236172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.236430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.236722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] PATH_INFO: `/v3/projects/813156a0534848c4b9eb730cc1838825/users/a672a95ef3594d61b8cf14d7b651b2ab/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.252742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813156a0534848c4b9eb730cc1838825, user_id=a672a95ef3594d61b8cf14d7b651b2ab, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.255525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.271676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1254/2516] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:44 2022] PUT /identity/v3/projects/813156a0534848c4b9eb730cc1838825/users/a672a95ef3594d61b8cf14d7b651b2ab/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:44.275866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:44.287391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152c161-662d-4955-8cd3-06a56ac8ce88 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.347125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.347837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.348127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.348409 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.349147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.350935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0cc39-9d47-4f53-9e40-50e75ce5cd44 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.366381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1263/2517] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:17:44.370662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:44.382329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77201bcb-10cb-4d49-b046-6dd802e5f4e9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:44.386965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:44.387903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:44.388186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:44.388456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] PATH_INFO: `/v3/projects/813156a0534848c4b9eb730cc1838825/users/a672a95ef3594d61b8cf14d7b651b2ab/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:44.390433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813156a0534848c4b9eb730cc1838825, user_id=a672a95ef3594d61b8cf14d7b651b2ab, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:44.393778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:44.405288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1255/2518] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:17:44 2022] PUT /identity/v3/projects/813156a0534848c4b9eb730cc1838825/users/a672a95ef3594d61b8cf14d7b651b2ab/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:17:49.994175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033013-b7bb-456f-8061-26367436beaf None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:49.994742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033013-b7bb-456f-8061-26367436beaf None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:49.994840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033013-b7bb-456f-8061-26367436beaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:50.023160 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-23033013-b7bb-456f-8061-26367436beaf None None] MFA Rules not processed for user `a672a95ef3594d61b8cf14d7b651b2ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:50.024618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-23033013-b7bb-456f-8061-26367436beaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:50.095307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1264/2519] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:17:49 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:50.120482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:50.122202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:50.214718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8d367-957c-4b8a-926b-0c7896b0c407 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:50.220569 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:50.221988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:50.221988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:50.221988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:50.291637 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:50.293213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:50.303739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1256/2520] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:17:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:17:52.784926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:52.785486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:52.786085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:52.820306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:17:52.824681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:52.875330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:52.919911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1265/2521] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:17:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:17:52.933161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:17:52.936568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:17:52.948472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:17:53.043137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc43c6-e83e-4b38-b581-6ba77f913ee4 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:17:53.049871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:17:53.050732 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:17:53.051328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:17:53.051986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:17:53.065503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:17:53.068543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77589277-612c-4497-8bf1-bbf7f1269eb8 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:17:53.090393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1257/2522] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:18:02.911909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=12 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:03.081105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=29 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:10.550085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:10.551806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:10.676911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3257f-e272-48dc-9d43-1448830f7305 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:10.682579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:10.683301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:10.683653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:10.683995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:10.686015 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:10.703119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:10.703119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:10.721294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1266/2523] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:17.145825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546665-0d5c-48c5-91e1-0e2b755e4d95 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:17.146366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546665-0d5c-48c5-91e1-0e2b755e4d95 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:17.146461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86546665-0d5c-48c5-91e1-0e2b755e4d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:17.165458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-86546665-0d5c-48c5-91e1-0e2b755e4d95 None None] MFA Rules not processed for user `e269c6a656ff4721911ae25657eeedca`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:17.166675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-86546665-0d5c-48c5-91e1-0e2b755e4d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:17.173997 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1258/2524] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:17.996675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:17.998965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:18.071847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba71287-da4a-4d8c-b6dc-ad17e60fc225 service placement] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:18.077148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:18.078155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:18.078475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:18.078757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] PATH_INFO: `/v3/users/058514d6c0594cf5887389c1838d3758` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:18.080113 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] RBAC: Authorizing `identity:delete_user(user_id=058514d6c0594cf5887389c1838d3758)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:18.082039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:18.162428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058514d6c0594cf5887389c1838d3758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:18.186174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058514d6c0594cf5887389c1838d3758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:18.192803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:18.203595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058514d6c0594cf5887389c1838d3758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:18.205375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058514d6c0594cf5887389c1838d3758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:18.206050 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2f2a531a-85c3-4ac4-9f6a-7af2b6e4424d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058514d6c0594cf5887389c1838d3758'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:18.221642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:18.222173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:18.222292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:18.240393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1267/2525] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:18:17 2022] DELETE /identity/v3/users/058514d6c0594cf5887389c1838d3758 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:18:18.242696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:18.244558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:18.244944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:18.249656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:18.251579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:18.281275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:18.286421 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:18.382631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1268/2526] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:18:18.400922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:18.404211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:18.410476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1259/2527] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:18:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:18.451288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:18.452884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:18.691346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee6b0d-89df-4d94-acc7-f8d97b444eb5 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:18.697706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287af16d-580a-48ff-8fa3-6c16452aa7f6 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:18.700333 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:18.701028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:18.701316 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:18.701635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:18.713536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:18.714947 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:18.718130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:18.718683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:18.719368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:18.721079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:18.739006 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1269/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:18.754390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:18.763848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd126cb-ab95-4ec0-8adb-329a35e453b6 service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:18.788819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1260/2529] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 04:18:19.142280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.144363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:19.160982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c85ec7-24ca-4b53-9477-9d122dcfecb2 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.168401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.169633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.170094 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.170551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] PATH_INFO: `/v3/projects/e277bba43c6a42218baa2f57f67298fc` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.172052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e277bba43c6a42218baa2f57f67298fc)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.174407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.245948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e277bba43c6a42218baa2f57f67298fc', 'request_id': 'req-663ae274-9f98-4f99-a216-10462a96ca77'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:19.256973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e277bba43c6a42218baa2f57f67298fc', 'request_id': 'req-663ae274-9f98-4f99-a216-10462a96ca77'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:19.264090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e277bba43c6a42218baa2f57f67298fc', 'request_id': 'req-663ae274-9f98-4f99-a216-10462a96ca77'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:19.273181 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:19.279066 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] The token cache is being invalidate because project e277bba43c6a42218baa2f57f67298fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:18:19.279744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:19.281284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-663ae274-9f98-4f99-a216-10462a96ca77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:18:19.283876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1270/2530] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:18:19 2022] DELETE /identity/v3/projects/e277bba43c6a42218baa2f57f67298fc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:18:19.290715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.291219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.291599 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.349074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:19.351137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:19.410091 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:19.420543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1261/2531] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:19.428478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.428771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.429195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.447319 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:19.448462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:19.459145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1271/2532] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:18:19.461320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.471377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aca0d3-d644-4c7a-844d-25d1bb600882 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.476539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.477206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.477496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.477778 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.478537 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:18:19.478851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.480487 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.493179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1262/2533] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:18:19.497479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.507127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d37f4-0ca5-4f21-aac6-155db7db4b7c None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.513454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.514156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.514440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.514720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.515465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.517375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.547140 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1272/2534] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:18:19 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:18:19.551103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.562543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5c9e6-14cb-456d-9000-98a822de9572 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.631121 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.631905 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.632314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.632755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.633547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.635565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.655899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1263/2535] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:18:19 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:18:19.660401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.671141 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf56600-8be6-4b8e-a798-a751c1c1a7f0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.676096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.676760 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.677075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.677335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.678091 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.679920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.691207 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1273/2536] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:18:19.695748 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.711170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb677a05-418c-441b-adde-747e7bc983d0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.716690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.717361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.717646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.717924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.718651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.720294 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810db92-2cb4-496b-b0d9-5ff93e254007 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.730827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1264/2537] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:18:19.735036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.745878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af47ccc-5e88-4565-ba71-a3aa9b25f087 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.752414 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.753344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.753620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.753910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] PATH_INFO: `/v3/projects/bf207393082c48d09f827e16d99b2609/users/17540f6ae3234debb5cfe7580f78d61d/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.770003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf207393082c48d09f827e16d99b2609, user_id=17540f6ae3234debb5cfe7580f78d61d, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.772791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.788131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1274/2538] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:18:19 2022] PUT /identity/v3/projects/bf207393082c48d09f827e16d99b2609/users/17540f6ae3234debb5cfe7580f78d61d/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:18:19.794087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf7c1-ca9d-4779-8031-0f78054caa08 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.794382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf7c1-ca9d-4779-8031-0f78054caa08 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.794683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cf7c1-ca9d-4779-8031-0f78054caa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.812788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6e9cf7c1-ca9d-4779-8031-0f78054caa08 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:19.814004 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6e9cf7c1-ca9d-4779-8031-0f78054caa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:19.878608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1265/2539] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:19.888492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:19.890315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:19.949674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691c847-52f3-4986-bb54-4914e63297d9 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:19.956189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:19.957096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:19.957471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:19.957835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:19.974350 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:19.976392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f52796-5081-42b7-bc81-bf305c0a9c8a service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:19.986546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1275/2540] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:21.745108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:21.746392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:21.746688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:21.765830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] MFA Rules not processed for user `abfd09ef27184291a36b43b6664740c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:21.767752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:21.855952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1266/2541] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:25.356461 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:25.356965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:25.357584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:25.402455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] MFA Rules not processed for user `17540f6ae3234debb5cfe7580f78d61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:25.404397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:25.518966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1276/2542] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:25.525768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:25.540056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6407e-f03b-415d-b36a-d05af316d6ef None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:25.558791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:25.559821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:25.560388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:25.561009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:25.562089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:25.576089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:25.587768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:25.624871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1267/2543] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:18:25 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:18:25.629022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:25.641388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1618bf82-8e3f-4c54-bd93-8000d5796eba None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:25.673462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:25.714296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:25.715041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:25.715463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:25.715802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:25.716780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:25.718557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:25.748001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1277/2544] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:18:25 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:18:25.749417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:25.769812 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5598e8-bb15-4de1-be25-d611a46c9c1a admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:25.782340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:25.783253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:25.783364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:25.783590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:25.784319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:25.785952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:25.799820 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1268/2545] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:18:25.807812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:25.833882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb18b21-9779-4985-ba4d-5f1c0421cb9d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:25.841657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:25.842942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:25.842942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:25.842942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] PATH_INFO: `/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c/users/8c6c8d69367a4e5d86a77195bcfb6e96/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:25.890596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45344bd3e7e64b76a20d9a98cd61bb4c, user_id=8c6c8d69367a4e5d86a77195bcfb6e96, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:25.893656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:25.922840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1278/2546] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:18:25 2022] PUT /identity/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c/users/8c6c8d69367a4e5d86a77195bcfb6e96/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:18:25.929145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:25.952709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44b704-6b74-46cd-a9a1-18af8487f57b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:26.057591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:26.058628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:26.059008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:26.059510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:26.060470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:26.062416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eda312-7eda-40c3-97c8-c6727835ec31 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:26.093379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1269/2547] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:18:26.096455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:26.119274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71ad9e-e77b-4e6f-8cf9-5fdf21380992 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:26.126944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:26.129247 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:26.129834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:26.130130 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] PATH_INFO: `/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c/users/8c6c8d69367a4e5d86a77195bcfb6e96/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:26.133873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45344bd3e7e64b76a20d9a98cd61bb4c, user_id=8c6c8d69367a4e5d86a77195bcfb6e96, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:26.136761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:26.151567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1279/2548] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:18:26 2022] PUT /identity/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c/users/8c6c8d69367a4e5d86a77195bcfb6e96/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:18:32.575019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1801fe-8778-435f-afda-9495bd70e255 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:32.575508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1801fe-8778-435f-afda-9495bd70e255 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:32.575649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1801fe-8778-435f-afda-9495bd70e255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:32.606902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8d1801fe-8778-435f-afda-9495bd70e255 None None] MFA Rules not processed for user `8c6c8d69367a4e5d86a77195bcfb6e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:32.608167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8d1801fe-8778-435f-afda-9495bd70e255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:32.692166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1270/2549] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:32.704253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:32.706107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:32.796835 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06f4bb-2fbd-44f7-a58f-6aa8745ebb5c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:32.800940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:32.801690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:32.802051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:32.802337 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:32.888020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:32.890985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:32.910873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1280/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:33.047962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f556-2857-439b-aff3-a7681e9655b1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:33.048345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f556-2857-439b-aff3-a7681e9655b1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:33.048725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364f556-2857-439b-aff3-a7681e9655b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:33.137318 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a364f556-2857-439b-aff3-a7681e9655b1 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:33.139218 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a364f556-2857-439b-aff3-a7681e9655b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:33.199676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:33.247827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1271/2551] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:18:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:33.262177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:33.263708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:33.284229 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:33.404631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72e4b8-e467-41bb-ba8f-0ca0982ca1d8 service glance] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:33.410008 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:33.410839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:33.411147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:33.411465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:33.420902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:33.422525 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:33.431778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1281/2552] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:18:42.481224 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a676d7-35de-4a8c-8853-7a8f5213d319 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:42.481761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a676d7-35de-4a8c-8853-7a8f5213d319 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:42.481875 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a676d7-35de-4a8c-8853-7a8f5213d319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:42.501421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-91a676d7-35de-4a8c-8853-7a8f5213d319 None None] MFA Rules not processed for user `b6854699a1dc475ab11a89bcaf2af85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:42.502691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-91a676d7-35de-4a8c-8853-7a8f5213d319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:42.589558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1272/2553] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:42.970352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:42.972451 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:43.053230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f2ac3-283c-4a0e-968c-7b4adbebeacc service swift] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:43.057432 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:43.058723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:43.058723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:43.058723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:43.063252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:43.148980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:43.150457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:43.162190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1282/2554] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:44.773012 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d888-10fe-4af8-a72e-0a97dab05e86 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:44.773912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d888-10fe-4af8-a72e-0a97dab05e86 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:44.774200 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6d888-10fe-4af8-a72e-0a97dab05e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:44.792910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-51a6d888-10fe-4af8-a72e-0a97dab05e86 None None] MFA Rules not processed for user `17540f6ae3234debb5cfe7580f78d61d`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:44.794133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-51a6d888-10fe-4af8-a72e-0a97dab05e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:44.819299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1273/2555] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:18:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:45.268476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:45.279257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd24ce-1136-4f57-8083-e90b05c6395c service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:45.286148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:45.286840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:45.287150 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:45.287434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:45.289584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:45.330174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=100 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:45.367394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:45.369012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:45.392878 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1283/2556] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:18:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:18:48.004579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dce6e3-5c0e-4985-9fde-51a462d02605 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:48.005622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dce6e3-5c0e-4985-9fde-51a462d02605 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:48.006125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dce6e3-5c0e-4985-9fde-51a462d02605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:48.048336 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-50dce6e3-5c0e-4985-9fde-51a462d02605 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:18:48.052410 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-50dce6e3-5c0e-4985-9fde-51a462d02605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:18:48.078039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1274/2557] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:18:48.092200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:18:48.119966 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f83a28-a587-41fd-9326-ab0156e68e8d service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:18:48.126514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:18:48.127397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:18:48.127977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:18:48.128402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:18:48.139592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:18:48.141614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:18:48.155576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1284/2558] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:18:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:18:58.030146 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=45 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:18:58.148365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=21 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:23.149368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:23.149949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:23.149949 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:23.168514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] MFA Rules not processed for user `8e3bc7e015f5419aa956568707764f67`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:19:23.169833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:23.230787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1275/2559] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:19:30.081687 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:30.092845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc83123-79ea-412d-ba74-a523e17c3a96 service swift] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:30.096990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:30.097708 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:30.098060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:30.098400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:30.107753 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:30.109272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:30.126013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1285/2560] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:19:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:19:33.223186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=25 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:40.114855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=6 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:45.610215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:45.611819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:45.681733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a988c4c-f1e0-4cfc-833b-1fbcce1de1aa None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:45.686648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:45.687581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:45.687861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:45.688140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] PATH_INFO: `/v3/users/abfd09ef27184291a36b43b6664740c5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:45.689328 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfd09ef27184291a36b43b6664740c5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:45.691568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:45.738057 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfd09ef27184291a36b43b6664740c5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:45.744741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfd09ef27184291a36b43b6664740c5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:45.759369 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfd09ef27184291a36b43b6664740c5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:45.760455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfd09ef27184291a36b43b6664740c5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:45.772639 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-eccc9dec-6c76-4156-b52d-1bdceb015780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfd09ef27184291a36b43b6664740c5'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:45.779619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1276/2561] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:19:45 2022] DELETE /identity/v3/users/abfd09ef27184291a36b43b6664740c5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:19:45.967685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:45.979821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb1d9-f672-4b21-b983-cf3b32bb49b2 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.038063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.038948 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.039255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.039538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] PATH_INFO: `/v3/projects/4a5e183a36084c9fb82f3a3f7d24402a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.040725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5e183a36084c9fb82f3a3f7d24402a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.042646 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.089342 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5e183a36084c9fb82f3a3f7d24402a', 'request_id': 'req-533f2422-301a-421d-acc8-f3f767fbca29'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:46.097865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5e183a36084c9fb82f3a3f7d24402a', 'request_id': 'req-533f2422-301a-421d-acc8-f3f767fbca29'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:46.104311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5e183a36084c9fb82f3a3f7d24402a', 'request_id': 'req-533f2422-301a-421d-acc8-f3f767fbca29'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:46.118454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] The token cache is being invalidate because project 4a5e183a36084c9fb82f3a3f7d24402a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:19:46.118740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:46.119430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-533f2422-301a-421d-acc8-f3f767fbca29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:19:46.121022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1286/2562] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:19:45 2022] DELETE /identity/v3/projects/4a5e183a36084c9fb82f3a3f7d24402a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:19:46.126385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.126684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.127023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.147337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:19:46.148539 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:46.214362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1277/2563] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:19:46.222378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.222670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.222939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.243251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:19:46.244438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:46.253780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1287/2564] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:19:46.258597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.273188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af374f2-f8d4-48e2-9dbf-d79654b93363 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.280084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.280753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.281062 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.281345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.282072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:19:46.282429 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.284347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.297093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1278/2565] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:19:46.301664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.311980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b1168-2474-42b8-969d-4bfdf740a67b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.318891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.319588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.319876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.320151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.320863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.322810 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.355405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1288/2566] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:19:46 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:19:46.356663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.367127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2548f-bc7e-42ba-9268-e3f4ff6257ee admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.371300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:46.434937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.435652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.435939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.436230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.436920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.438866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.462223 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1279/2567] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:19:46 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:19:46.466466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.477482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dba8cf-2495-4b0b-98a2-008ab40cb673 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.482585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.483301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.483621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.483906 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.484643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.486267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.498105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1289/2568] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:19:46.502434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.513071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ee9430-db31-4634-b14d-273b0196c2ca admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.518147 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.518858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.519148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.519447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.520191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.521797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c412b-c046-4a0b-abb6-552f55de1d94 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.532304 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1280/2569] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:19:46.536980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.546958 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e56bcc-18d8-4d3c-9e96-5afbc5ef1385 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.552607 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.553492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.553769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.554070 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] PATH_INFO: `/v3/projects/b0253934e99b48ce956fbe1ab3a7cbab/users/eb538ffb27fe4564b3064d11e5abbfc7/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.569283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0253934e99b48ce956fbe1ab3a7cbab, user_id=eb538ffb27fe4564b3064d11e5abbfc7, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.572044 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.587699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1290/2570] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:19:46 2022] PUT /identity/v3/projects/b0253934e99b48ce956fbe1ab3a7cbab/users/eb538ffb27fe4564b3064d11e5abbfc7/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:19:46.598211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.598582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.598941 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.625841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] MFA Rules not processed for user `eb538ffb27fe4564b3064d11e5abbfc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:19:46.626983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:46.694149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1281/2571] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:19:46.705465 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:19:46.707819 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:19:46.740661 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:46.792484 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529110d-fb0d-4028-b3c7-77a7bf410fb7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:19:46.796424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:19:46.797072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:19:46.797356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:19:46.797633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:19:46.806718 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:19:46.808180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a6af20-58f8-4468-adea-c07dd262750c service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:19:46.817472 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1291/2572] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:19:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:19:56.690806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:19:56.813201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=23 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:03.181590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:03.247730 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5fbf6e-ee59-42de-aa8b-939758cd3db2 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:03.253204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:03.254099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:03.254384 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:03.254677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] PATH_INFO: `/v3/users/eb538ffb27fe4564b3064d11e5abbfc7` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:03.255958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb538ffb27fe4564b3064d11e5abbfc7)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:03.257903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:03.310430 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb538ffb27fe4564b3064d11e5abbfc7'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.317332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb538ffb27fe4564b3064d11e5abbfc7'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.330956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb538ffb27fe4564b3064d11e5abbfc7'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.332183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb538ffb27fe4564b3064d11e5abbfc7'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.341768 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb538ffb27fe4564b3064d11e5abbfc7'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.348177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1282/2573] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:20:03 2022] DELETE /identity/v3/users/eb538ffb27fe4564b3064d11e5abbfc7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:20:03.352921 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:03.353430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:03.353538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:03.372295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:03.373504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:03.453230 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1292/2574] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:03.463075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:03.464706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:03.525725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00648a26-67f6-43c5-ac30-0e4cc54ff853 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:03.530491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:03.531130 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:03.531448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:03.531719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:03.541727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:03.543230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a01ac-bf48-4906-9f40-a3aa9364742a service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:03.553967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1283/2575] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:03.841977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:03.843427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:03.855885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e1e6a-47de-46b4-ad6a-12d8470c2978 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:03.861380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:03.862272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:03.862554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:03.862829 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] PATH_INFO: `/v3/projects/b0253934e99b48ce956fbe1ab3a7cbab` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:03.864144 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0253934e99b48ce956fbe1ab3a7cbab)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:03.866074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:03.917809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0253934e99b48ce956fbe1ab3a7cbab', 'request_id': 'req-f3652593-a092-4a18-aade-34719d49e95a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.926587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0253934e99b48ce956fbe1ab3a7cbab', 'request_id': 'req-f3652593-a092-4a18-aade-34719d49e95a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.932523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0253934e99b48ce956fbe1ab3a7cbab', 'request_id': 'req-f3652593-a092-4a18-aade-34719d49e95a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.943590 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] The token cache is being invalidate because project b0253934e99b48ce956fbe1ab3a7cbab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:20:03.943910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.944533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-f3652593-a092-4a18-aade-34719d49e95a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:03.946081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1293/2576] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:20:03 2022] DELETE /identity/v3/projects/b0253934e99b48ce956fbe1ab3a7cbab => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:20:03.951256 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:03.951548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:03.951828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:03.970075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:03.971307 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:03.988801 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:04.039519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1284/2577] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:04.103955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.104193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.104513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.125014 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:04.126235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:04.134890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1294/2578] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:04.139872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.154438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba078dc-7ce3-4f34-9867-c7fb33f860fa None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.160404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.161065 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.161344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.161603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.162316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:20:04.162622 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.164263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.176536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1285/2579] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:20:04.180940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.190559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa7fb9-bf70-45e6-8a48-472bc12ae106 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.196548 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.197245 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.197526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.197801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.198492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.200523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.230010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1295/2580] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:20:04 2022] POST /identity/v3/projects => generated 399 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:20:04.233895 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.246511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88ead-d170-4756-b443-a3b91827c1fc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.318044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.318751 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.319040 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.319342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.320042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.321961 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.344710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1286/2581] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:20:04 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:20:04.348864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.360090 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0542cdf-225c-4a2d-89ba-e34028e6646f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.364702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.365368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.365652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.365930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.366663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.368293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.379318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1296/2582] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:20:04.383448 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.394315 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9465738-7eaf-417f-88fe-d4b557924be6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.398976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.399681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.399964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.400240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.400972 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.402654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3c2a7-8d13-492a-be76-3dcba9a44717 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.413248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1287/2583] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:20:04.417531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.428374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00205c-4b85-4d78-9070-1eeb7019c8ac admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.434067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.435007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.435375 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.435663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] PATH_INFO: `/v3/projects/5f0878868d104652b4c5cf5bd2022a4a/users/e6fb74317f154ebd834b19c9a27564b4/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.451545 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f0878868d104652b4c5cf5bd2022a4a, user_id=e6fb74317f154ebd834b19c9a27564b4, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.454215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.472711 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1297/2584] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:20:04 2022] PUT /identity/v3/projects/5f0878868d104652b4c5cf5bd2022a4a/users/e6fb74317f154ebd834b19c9a27564b4/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:20:04.485955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.486180 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.486463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.516498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] MFA Rules not processed for user `e6fb74317f154ebd834b19c9a27564b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:04.517711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:04.586140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1288/2585] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:20:04.598299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:04.600348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:04.623800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:04.661653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2e80b-714a-499b-9cdc-840391067f56 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:04.665863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:04.666508 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:04.666795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:04.667081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:04.676416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:04.677846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e65379-98ad-4d70-b19e-139c99078a23 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:04.686792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1298/2586] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:04 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:11.798766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:11.825033 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:11.874309 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca6065-158e-4c91-aa9c-521d949b8735 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:11.879436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:11.880334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:11.880624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:11.880909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] PATH_INFO: `/v3/users/e6fb74317f154ebd834b19c9a27564b4` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:11.882129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6fb74317f154ebd834b19c9a27564b4)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:11.884105 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:11.934942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6fb74317f154ebd834b19c9a27564b4'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:11.942673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6fb74317f154ebd834b19c9a27564b4'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:11.956152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6fb74317f154ebd834b19c9a27564b4'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:11.956920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6fb74317f154ebd834b19c9a27564b4'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:11.965524 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6fb74317f154ebd834b19c9a27564b4'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:11.972220 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1289/2587] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:20:11 2022] DELETE /identity/v3/users/e6fb74317f154ebd834b19c9a27564b4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:20:11.977039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:11.977546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:11.977655 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:11.995566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:11.996789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:12.061017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1299/2588] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:12.071053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:12.072756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:12.133833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d60bf-c751-44ca-bb2f-041aaa607183 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:12.138183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:12.139156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:12.139447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:12.139722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:12.150195 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:12.151689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:12.161682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1290/2589] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:12.458768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:12.460271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:12.473369 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26422a-fa6c-4cb3-bc48-7e6e10eb9827 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:12.479396 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:12.480260 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:12.480561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:12.480836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] PATH_INFO: `/v3/projects/5f0878868d104652b4c5cf5bd2022a4a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:12.482063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f0878868d104652b4c5cf5bd2022a4a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:12.484046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:12.537887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f0878868d104652b4c5cf5bd2022a4a', 'request_id': 'req-7b293422-89ed-4446-ba11-e63c7e835a85'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:12.546324 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f0878868d104652b4c5cf5bd2022a4a', 'request_id': 'req-7b293422-89ed-4446-ba11-e63c7e835a85'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:12.552108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f0878868d104652b4c5cf5bd2022a4a', 'request_id': 'req-7b293422-89ed-4446-ba11-e63c7e835a85'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:12.563877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] The token cache is being invalidate because project 5f0878868d104652b4c5cf5bd2022a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:20:12.564165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:12.564822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:20:12.566464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1300/2590] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:20:12 2022] DELETE /identity/v3/projects/5f0878868d104652b4c5cf5bd2022a4a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:20:13.460735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:13.462148 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:13.505786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:13.522915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf9001-d824-4d8f-9afd-f02c244467fe service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:13.580382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:13.581061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:13.581308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:13.582225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:13.583650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:13.672657 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:13.674026 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:13.695492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1291/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:18.151374 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:18.153689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:18.210733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:18.218003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b293422-89ed-4446-ba11-e63c7e835a85 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:18.223040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:18.223729 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:18.224019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:18.224299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:18.226131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:18.240855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:18.242296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:18.253637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1301/2592] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:21.981652 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:21.983867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:21.998392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305edaa-a9dc-47be-b3bc-fa6c981333e0 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:22.005582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:22.006286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:22.006562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:22.006839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:22.008901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:22.070934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:22.072594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2b498-6475-4436-a9ac-87a95626880c service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:22.083700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1292/2593] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:28.248601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:32.078323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=93 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:38.711728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:38.720780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d837bf-c6c3-418b-a0b5-e7eb1bb95f34 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:38.725837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:38.726485 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:38.726820 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:38.727041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:38.729147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:38.739666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:38.741084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:38.749022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1302/2594] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:40.247428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba54b0c-bd47-4c37-ab64-f7e49c22bfa5 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:40.247888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba54b0c-bd47-4c37-ab64-f7e49c22bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:40.248202 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba54b0c-bd47-4c37-ab64-f7e49c22bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:40.270334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7ba54b0c-bd47-4c37-ab64-f7e49c22bfa5 None None] MFA Rules not processed for user `b6854699a1dc475ab11a89bcaf2af85a`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:40.271817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7ba54b0c-bd47-4c37-ab64-f7e49c22bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:40.332963 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1293/2595] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:20:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:44.361215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:44.363020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:44.381289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60da36-9937-45c0-b554-9b55c00d241a service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:44.390934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:44.391663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:44.391913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:44.392192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:44.394118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:44.409441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:44.411885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:44.423187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1303/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:47.762095 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:47.762555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:47.762661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:47.781542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:20:47.782737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:47.841391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1294/2597] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:20:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:20:47.884846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:47.886487 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:47.964908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975cf73-99e0-4943-9edf-258aefca416f service nova] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:47.970677 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:47.971370 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:47.971658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:47.971941 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:47.980843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:47.982466 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:47.990689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1304/2598] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:20:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:20:49.098374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:20:49.116117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f65181-53b2-43ca-ab5b-996e412022c1 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:20:49.121868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:20:49.122542 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:20:49.122832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:20:49.123125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:20:49.125238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:20:49.140737 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:20:49.142281 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88599d-bd75-4298-94df-5a9ede85ffdb service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:20:49.157797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1295/2599] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:20:57.987737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=44 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:20:59.149702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=56 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:01.187782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:01.197978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd2fa1-76c1-4cf0-8608-e07a7a2fb1da service swift] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:01.202207 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:01.202788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.203079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.203470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.205373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:01.215643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:01.217080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec47324-6154-4d7a-aaee-d13bd866e8ec service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:01.225281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1305/2600] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:01.338439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.339014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.339014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.340405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1296/2601] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:21:01.346768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9b9d-adfd-4864-bfc0-ae4b61a3fb43 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.347054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9b9d-adfd-4864-bfc0-ae4b61a3fb43 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.347376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9b9d-adfd-4864-bfc0-ae4b61a3fb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.385569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ac8e9b9d-adfd-4864-bfc0-ae4b61a3fb43 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:01.386885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ac8e9b9d-adfd-4864-bfc0-ae4b61a3fb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:01.400210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1306/2602] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:21:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:21:01.413032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:01.414802 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:01.481299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573199ac-91fb-4786-a0b2-97e4e9b2c13d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:01.485158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:01.485786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.486073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.486349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.497158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:01.498607 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d7faf-e499-47ac-b65a-8b96f48f4551 service glance] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:01.510060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1297/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:01.585742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7e56d-ad2c-465f-a55b-fdacf00c1cbe None None] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.586030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7e56d-ad2c-465f-a55b-fdacf00c1cbe None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.586307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7e56d-ad2c-465f-a55b-fdacf00c1cbe None None] PATH_INFO: `/` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.587728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1307/2604] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:21:01.594556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b795-9e51-4749-b876-1d2e43036947 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.594846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b795-9e51-4749-b876-1d2e43036947 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.595126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9b795-9e51-4749-b876-1d2e43036947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.614626 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3ed9b795-9e51-4749-b876-1d2e43036947 None None] MFA Rules not processed for user `00a87fd4c488404f98aa12190f598cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:01.615868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3ed9b795-9e51-4749-b876-1d2e43036947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:01.622616 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1298/2605] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:21:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:01.635571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7e56d-ad2c-465f-a55b-fdacf00c1cbe None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:01.651584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7e56d-ad2c-465f-a55b-fdacf00c1cbe None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:01.656288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:01.656933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.657221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.657497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.664956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:01.666386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221d0480-e8da-42c3-b3ee-89c78adacab6 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:01.674301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1308/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:01.766427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869dfab-d801-412d-90ed-9b66dbb2086c None None] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.766719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869dfab-d801-412d-90ed-9b66dbb2086c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.766971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869dfab-d801-412d-90ed-9b66dbb2086c None None] PATH_INFO: `/` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.768485 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1299/2607] 10.222.0.13 () {54 vars in 921 bytes} [Thu Jun 30 04:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 04:21:01.774069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.774358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.774631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.794541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] MFA Rules not processed for user `67b4385cb5ed42d1b9488d409abd61b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:01.795789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:01.806972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1309/2608] 10.222.0.13 () {58 vars in 1018 bytes} [Thu Jun 30 04:21:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:01.821898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869dfab-d801-412d-90ed-9b66dbb2086c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:01.838053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869dfab-d801-412d-90ed-9b66dbb2086c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:01.841765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:01.842440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:01.842726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:01.843010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:01.854238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:01.855836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a481ff-5a0b-48e0-9587-91a2e00a6872 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:01.866764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1300/2609] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:01.937675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:01.939543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:02.013745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e958a-bc4a-45c0-b81f-d895ec595c50 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:02.020834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:02.021849 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:02.022271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:02.022683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:02.025129 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:02.038335 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:02.040393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:02.053595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1310/2610] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:03.710151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:03.710715 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:03.710860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:03.731383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:03.732760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:03.740488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1301/2611] 10.222.0.13 () {58 vars in 1009 bytes} [Thu Jun 30 04:21:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:03.752455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:03.764557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad88a-da9a-43af-b988-47146d5653f5 service placement] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:03.769488 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:03.770203 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:03.770489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:03.770784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:03.779757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:03.781704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1ac88-1fcb-4d0d-9baf-8c4bd9b664dc service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:03.789523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1311/2612] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:21:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:21:13.725729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=38 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:13.786238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=62 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:15.212161 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:15.232030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793e65-51f7-4500-a0e7-0084d00aefc1 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:15.238099 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:15.238817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:15.239137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:15.239418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:15.241349 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:15.256249 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:15.258098 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:15.270412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1302/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:22.890475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3310-acc5-433c-bfae-2c74ca44f82d None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:22.891410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3310-acc5-433c-bfae-2c74ca44f82d None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:22.891723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a3310-acc5-433c-bfae-2c74ca44f82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:22.914952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-414a3310-acc5-433c-bfae-2c74ca44f82d None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:22.916218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-414a3310-acc5-433c-bfae-2c74ca44f82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:22.923725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1312/2614] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:22.937848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:22.950319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff004f-5462-4048-92ba-fb239143b111 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:22.956246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:22.957007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:22.957300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:22.957585 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:22.967443 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:22.969058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1117f0-6ac4-458c-81c2-1542dd4628ec service swift] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:22.977421 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1303/2615] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:21:23.329121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:23.329408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:23.330238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:23.353700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] MFA Rules not processed for user `e269c6a656ff4721911ae25657eeedca`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:23.355743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:23.415402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1313/2616] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:24.553769 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:24.554258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:24.554343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:24.594045 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] MFA Rules not processed for user `8079221b336646d3af1dc7aba9719ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:24.595474 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:24.605828 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1304/2617] 10.222.0.13 () {58 vars in 967 bytes} [Thu Jun 30 04:21:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:24.619481 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:24.633601 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0d76e-6e67-4b37-af91-e60ec0efb51f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:24.639946 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0857fc0c4021456b88288ba10921ca6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:24.640618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:24.640907 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:24.641190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:24.650546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:24.652438 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:24.661951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1314/2618] 10.222.0.13 () {60 vars in 1384 bytes} [Thu Jun 30 04:21:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 04:21:28.734836 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:28.736500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:28.760242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd7437-fde0-4b5a-9b1c-a43c58a51d0c None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:28.766526 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:28.767324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:28.767561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:28.767876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:28.769928 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:28.785210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:28.787297 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:28.799983 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1305/2619] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:33.258629 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:33.260769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:33.373561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6d63b-81ef-4b32-94cb-70a6be24f5a4 service swift] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:33.378798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:33.379953 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:33.380473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:33.381176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] PATH_INFO: `/v3/users/17540f6ae3234debb5cfe7580f78d61d` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:33.382724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=17540f6ae3234debb5cfe7580f78d61d)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:33.384751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:33.441744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17540f6ae3234debb5cfe7580f78d61d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.454769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17540f6ae3234debb5cfe7580f78d61d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.462215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17540f6ae3234debb5cfe7580f78d61d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.463012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17540f6ae3234debb5cfe7580f78d61d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.463653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17540f6ae3234debb5cfe7580f78d61d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.480506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1315/2620] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:21:33 2022] DELETE /identity/v3/users/17540f6ae3234debb5cfe7580f78d61d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:33.669898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:33.686938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e47bb2-886e-4bae-983c-42b01180dd2f service placement] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:33.748770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:33.749653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:33.749938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:33.750234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] PATH_INFO: `/v3/projects/bf207393082c48d09f827e16d99b2609` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:33.751472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf207393082c48d09f827e16d99b2609)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:33.753398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:33.800531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf207393082c48d09f827e16d99b2609', 'request_id': 'req-769faf09-0707-4ba3-b2af-286274510b23'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.808508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf207393082c48d09f827e16d99b2609', 'request_id': 'req-769faf09-0707-4ba3-b2af-286274510b23'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.814317 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf207393082c48d09f827e16d99b2609', 'request_id': 'req-769faf09-0707-4ba3-b2af-286274510b23'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.825749 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] The token cache is being invalidate because project bf207393082c48d09f827e16d99b2609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:21:33.826043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.826725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.828306 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1306/2621] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:21:33 2022] DELETE /identity/v3/projects/bf207393082c48d09f827e16d99b2609 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:33.832356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:33.833911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:33.843884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:33.848426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875db8b-b07b-4c72-ae2d-f94a20213eda admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:33.911059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:33.912152 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:33.912527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:33.912815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] PATH_INFO: `/v3/users/8c6c8d69367a4e5d86a77195bcfb6e96` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:33.914018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6c8d69367a4e5d86a77195bcfb6e96)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:33.915978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:33.965205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6c8d69367a4e5d86a77195bcfb6e96'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.977505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6c8d69367a4e5d86a77195bcfb6e96'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.984504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6c8d69367a4e5d86a77195bcfb6e96'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.985197 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6c8d69367a4e5d86a77195bcfb6e96'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.985800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6c8d69367a4e5d86a77195bcfb6e96'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:33.999649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1316/2622] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:21:33 2022] DELETE /identity/v3/users/8c6c8d69367a4e5d86a77195bcfb6e96 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:34.199141 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:34.200954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:34.214441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769faf09-0707-4ba3-b2af-286274510b23 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:34.260686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:34.277980 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:34.278893 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:34.279216 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:34.279519 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] PATH_INFO: `/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:34.280740 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45344bd3e7e64b76a20d9a98cd61bb4c)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:34.282682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:34.335712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45344bd3e7e64b76a20d9a98cd61bb4c', 'request_id': 'req-9d5075eb-d499-4094-bf84-681bcf5ef643'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:34.346876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45344bd3e7e64b76a20d9a98cd61bb4c', 'request_id': 'req-9d5075eb-d499-4094-bf84-681bcf5ef643'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:34.353956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45344bd3e7e64b76a20d9a98cd61bb4c', 'request_id': 'req-9d5075eb-d499-4094-bf84-681bcf5ef643'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:34.365227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] The token cache is being invalidate because project 45344bd3e7e64b76a20d9a98cd61bb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:21:34.365518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:34.366364 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:34.368234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1307/2623] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:21:34 2022] DELETE /identity/v3/projects/45344bd3e7e64b76a20d9a98cd61bb4c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:43.322048 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:43.323617 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:43.401556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a9c4f-b928-48d5-8984-71cb6e903a64 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:43.420013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:43.479300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:43.480733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:43.480733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:43.481116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:43.483450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:43.544125 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:43.545728 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:43.555532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1317/2624] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:44.362198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:47.030940 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:47.032498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:47.097171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5075eb-d499-4094-bf84-681bcf5ef643 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:47.103194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:47.103863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:47.104151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:47.104446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:47.106379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:47.176810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:47.178289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:47.188647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1308/2625] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:52.051761 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.053352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:52.067742 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7503eeb-55df-46e6-9a69-cf79c022091f service nova] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.072791 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.073776 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.074060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.074357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] PATH_INFO: `/v3/users/b6854699a1dc475ab11a89bcaf2af85a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.075721 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6854699a1dc475ab11a89bcaf2af85a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.077660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.126047 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6854699a1dc475ab11a89bcaf2af85a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.137714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6854699a1dc475ab11a89bcaf2af85a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.144314 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6854699a1dc475ab11a89bcaf2af85a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.145068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6854699a1dc475ab11a89bcaf2af85a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.145795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-3f69e87f-a015-4e7e-98a5-043411602753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6854699a1dc475ab11a89bcaf2af85a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.160957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1318/2626] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:21:52 2022] DELETE /identity/v3/users/b6854699a1dc475ab11a89bcaf2af85a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:52.372966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.387228 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd56ea-3d1f-4044-9c9c-77645d11ea9c service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.446116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.446957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.447272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.447551 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] PATH_INFO: `/v3/projects/74b53f46d15c4465b9005939114e67a3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.448765 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b53f46d15c4465b9005939114e67a3)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.450692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.499533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b53f46d15c4465b9005939114e67a3', 'request_id': 'req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.502028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:52.509618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b53f46d15c4465b9005939114e67a3', 'request_id': 'req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.516441 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b53f46d15c4465b9005939114e67a3', 'request_id': 'req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.528210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] The token cache is being invalidate because project 74b53f46d15c4465b9005939114e67a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:21:52.528502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.529138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bb9fd02b-f10b-4ae0-8f13-2f84525249d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:21:52.530681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1309/2627] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:21:52 2022] DELETE /identity/v3/projects/74b53f46d15c4465b9005939114e67a3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:52.535936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.536255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.536533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.554511 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:52.555896 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:52.581972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:52.618983 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1319/2628] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:52.627373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.627689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.627950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.646417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:52.647668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:52.657849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1310/2629] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:52.662109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.672104 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad504aa0-ff33-4413-b283-054597a3c1ee None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.678416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.679059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.679367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.679643 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.680411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:21:52.680847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.683344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.701833 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1320/2630] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:21:52.706380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.716472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86863c-30d5-4234-a0a3-2c7413469b85 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.722340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.723005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.723340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.723600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.724291 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.726217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.767240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1311/2631] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:21:52 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:21:52.771937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.783770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4660c2-d33a-447c-9a52-8bc1266e4b3e admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.842473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.843158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.843467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.843737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.844431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.846336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.870227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1321/2632] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:21:52 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:21:52.871625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.884872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d59fee-d3a0-4919-b429-0e4843d3cbfd admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.889926 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.890615 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.890908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.891260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.892002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.893683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.905139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1312/2633] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:21:52.909455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.919965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc2f3e-7c5d-402c-b024-b722e5960f10 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.925712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.926477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.926758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.927032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.927783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.929401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df99f5a0-09f1-4529-a7f3-00beb1f9bbd7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.940554 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1322/2634] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:21:52.944835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:52.956084 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5384-b30f-4099-ab4e-6658f7968af9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:52.961833 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:52.962727 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:52.963022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:52.963324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] PATH_INFO: `/v3/projects/cec258de3eea4a12a7bd9b2b9832dce6/users/a432229e175149b0bc254f19af8007fa/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:52.979217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec258de3eea4a12a7bd9b2b9832dce6, user_id=a432229e175149b0bc254f19af8007fa, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:52.981886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:52.999043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1313/2635] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:21:52 2022] PUT /identity/v3/projects/cec258de3eea4a12a7bd9b2b9832dce6/users/a432229e175149b0bc254f19af8007fa/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:21:53.004353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f25c18-e0ac-489d-97fb-c0ccc50f602f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:53.004648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f25c18-e0ac-489d-97fb-c0ccc50f602f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:53.004934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f25c18-e0ac-489d-97fb-c0ccc50f602f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:53.023749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-52f25c18-e0ac-489d-97fb-c0ccc50f602f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:53.024771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-52f25c18-e0ac-489d-97fb-c0ccc50f602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:53.088840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1323/2636] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:53.098841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:53.100695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:53.167576 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacba4d4-d490-4f89-960c-3f25b6b78d30 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:53.172160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:53.172813 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:53.173094 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:53.173368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:53.183366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:53.184809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:53.194504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1314/2637] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:57.173512 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b768-cb03-475b-a7bc-07a1fb77f294 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:57.174069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b768-cb03-475b-a7bc-07a1fb77f294 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:57.174069 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422b768-cb03-475b-a7bc-07a1fb77f294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:57.208292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-0422b768-cb03-475b-a7bc-07a1fb77f294 None None] MFA Rules not processed for user `a432229e175149b0bc254f19af8007fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:57.209533 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-0422b768-cb03-475b-a7bc-07a1fb77f294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:57.239467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:57.293385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1324/2638] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:57.310543 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:57.314326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:57.419553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9aaa1-d161-411c-bd33-73789994d763 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:57.426247 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:57.426918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:57.427241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:57.427533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:57.434048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:21:57.436957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:57.438670 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:57.452440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1315/2639] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:21:58.919049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:58.919672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:58.919771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:58.938448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] MFA Rules not processed for user `a432229e175149b0bc254f19af8007fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:21:58.940262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:58.948743 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1325/2640] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:21:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:21:59.281390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:21:59.291681 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985ec9-686a-4676-b39d-8fa97be05b3c service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:21:59.295477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:21:59.296217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:21:59.296767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:21:59.298419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:21:59.300757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:21:59.311597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:21:59.313080 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:21:59.322602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1316/2641] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:21:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:22:05.730288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:05.741306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4645972-aac6-43a1-b0ae-b7bac7391e3e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:05.746543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:05.747292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:05.747586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:05.747837 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:05.749891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:05.763967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:05.765457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:05.777858 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1326/2642] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:09.242143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:09.244139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:09.258996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8df0d-5e6d-41ee-a4c1-58204ec8e44d service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:09.263561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:09.264619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.264962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.265370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] PATH_INFO: `/v3/users/8e3bc7e015f5419aa956568707764f67` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.266624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3bc7e015f5419aa956568707764f67)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:09.268809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:09.329386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3bc7e015f5419aa956568707764f67'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.337699 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3bc7e015f5419aa956568707764f67'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.355345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3bc7e015f5419aa956568707764f67'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.356172 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3bc7e015f5419aa956568707764f67'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.372041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-001e8017-2b13-4848-9a9e-b27143dd498b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3bc7e015f5419aa956568707764f67'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.378682 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1317/2643] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:09 2022] DELETE /identity/v3/users/8e3bc7e015f5419aa956568707764f67 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:09.570301 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:09.581460 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efd51da-b6e9-4f7f-b9a4-149383c2f795 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:09.648102 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:09.649007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.649770 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.650054 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] PATH_INFO: `/v3/projects/71588f8a5fc34919a77f28b6a8c8db0e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.651630 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71588f8a5fc34919a77f28b6a8c8db0e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:09.653531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:09.702647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71588f8a5fc34919a77f28b6a8c8db0e', 'request_id': 'req-caf122d9-34cf-4306-a248-102662831cc3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.711077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71588f8a5fc34919a77f28b6a8c8db0e', 'request_id': 'req-caf122d9-34cf-4306-a248-102662831cc3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.717120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71588f8a5fc34919a77f28b6a8c8db0e', 'request_id': 'req-caf122d9-34cf-4306-a248-102662831cc3'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.728117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] The token cache is being invalidate because project 71588f8a5fc34919a77f28b6a8c8db0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:09.728403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.729200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-caf122d9-34cf-4306-a248-102662831cc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:09.730771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1327/2644] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:09 2022] DELETE /identity/v3/projects/71588f8a5fc34919a77f28b6a8c8db0e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:09.735922 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.736217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.736502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.754645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:09.755887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:09.818921 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1318/2645] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:09.826956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.827283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.827559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.845935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:09.847133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:09.855592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1328/2646] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:09.860446 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:09.871134 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c68802-b114-4612-a2c9-7205bdb1da72 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:09.876299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:09.876970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.877260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.877536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.878272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:09.878582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:09.880234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:09.891969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1319/2647] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:09.896427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:09.906658 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae93ef-d980-43c6-b23a-04f7a32d0ff7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:09.912115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:09.912769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:09.913036 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:09.913328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:09.914029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:09.916634 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:09.944886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1329/2648] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:09 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:09.949059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:09.959651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360f35d-973d-4db6-af93-8080a5e29b96 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:10.003169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:10.022844 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:10.023586 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.023874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.024211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.025100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:10.027222 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:10.053874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1320/2649] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:09 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:10.058165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:10.068874 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f89b39-ee0e-4d49-bb6a-e0f12a46c403 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:10.074263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:10.074884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.075195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.075477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.076202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:10.077815 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:10.080159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:10.093046 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1330/2650] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:10.094884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:10.106213 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ae48-bc4d-4623-8224-9036fb09a44d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:10.111055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:10.111649 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.111935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.112215 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.112954 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:10.114562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14637065-179b-4bac-9c91-9b93dd342f29 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:10.131053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1321/2651] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:10.131198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:10.141700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1737715-37c7-45d7-9943-0c17d668940c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:10.146600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:10.147542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.147831 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.148112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] PATH_INFO: `/v3/projects/f9005946803a45dba07c41408684dd7a/users/d54caea77ad846689e836cca3a33a754/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.164237 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9005946803a45dba07c41408684dd7a, user_id=d54caea77ad846689e836cca3a33a754, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:10.167551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:10.184035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1331/2652] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:10 2022] PUT /identity/v3/projects/f9005946803a45dba07c41408684dd7a/users/d54caea77ad846689e836cca3a33a754/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:10.194710 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.194999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.195280 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.224445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] MFA Rules not processed for user `d54caea77ad846689e836cca3a33a754`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:10.225658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:10.296077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1322/2653] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:10.307427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:10.309226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:10.384284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca3ecb-de8f-4586-8328-7ca10792b31d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:10.389620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:10.390286 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:10.390569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:10.390842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:10.400600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:10.402377 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ef4886-fc64-41f4-9074-21b840319b7d service nova] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:10.414871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1332/2654] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:13.280965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:13.365834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc731af-f5e4-4521-a0ab-78d9a906760a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:13.372927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:13.374332 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:13.374784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:13.375205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] PATH_INFO: `/v3/users/d54caea77ad846689e836cca3a33a754` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:13.376534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54caea77ad846689e836cca3a33a754)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:13.378491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:13.437575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54caea77ad846689e836cca3a33a754'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:13.444205 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:13.447662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54caea77ad846689e836cca3a33a754'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:13.465978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54caea77ad846689e836cca3a33a754'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:13.467170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54caea77ad846689e836cca3a33a754'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:13.480452 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54caea77ad846689e836cca3a33a754'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:13.486825 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1323/2655] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:13 2022] DELETE /identity/v3/users/d54caea77ad846689e836cca3a33a754 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:13.491458 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:13.491963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:13.492219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:13.511343 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:13.512571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:13.574538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1333/2656] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:13.584401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:13.585902 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:13.643835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691f2fd-31b0-4642-80bb-9d21a2253efb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:13.647835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:13.648491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:13.648780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:13.649059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:13.658978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:13.660459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70c6e2-f73a-4cc2-b2cc-22e073c2a323 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:13.669987 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1324/2657] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:13.963449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:13.965120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:13.977767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5df80-4c70-4e02-a98d-3a6e897f0a2e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:13.983606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:13.984506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:13.984792 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:13.985074 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] PATH_INFO: `/v3/projects/f9005946803a45dba07c41408684dd7a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:13.986385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9005946803a45dba07c41408684dd7a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:13.988380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.046529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9005946803a45dba07c41408684dd7a', 'request_id': 'req-d931d0a2-bf3b-439e-a8d4-6f105920102d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:14.051970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:14.060923 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9005946803a45dba07c41408684dd7a', 'request_id': 'req-d931d0a2-bf3b-439e-a8d4-6f105920102d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:14.068780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9005946803a45dba07c41408684dd7a', 'request_id': 'req-d931d0a2-bf3b-439e-a8d4-6f105920102d'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:14.082491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] The token cache is being invalidate because project f9005946803a45dba07c41408684dd7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:14.082779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:14.083573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-d931d0a2-bf3b-439e-a8d4-6f105920102d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:14.085238 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1334/2658] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:13 2022] DELETE /identity/v3/projects/f9005946803a45dba07c41408684dd7a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:14.091581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.091899 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.092140 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.114145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:14.115381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:14.185186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1325/2659] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:14.194164 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.194457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.194793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.213366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:14.214579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:14.222690 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1335/2660] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:14.227087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.237561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c8433-10a9-4d17-a94f-f30f1721410e None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.242766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.243594 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.243925 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.244319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.245067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:14.245377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.247126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.259892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1326/2661] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:14.264407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.273863 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31590f4a-4edf-43d7-a88e-291397b4fa3a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.279359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.280004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.280288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.280567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.281267 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.283248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.310217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1336/2662] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:14.314490 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.325151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb184d-0005-4faf-a79f-46425a8cbf30 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.384398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.385192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.385486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.385741 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.386458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.388431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.395444 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:14.411571 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1327/2663] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:22:14.416086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.427505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83224f1a-fc46-4d8c-8339-a0bb02380d84 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.433653 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.434613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.434955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.435264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.436026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.437631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.449529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1337/2664] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:14.453989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.465036 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f93bb-1055-4b81-8c43-87812ba1998b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.471072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.471754 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.472039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.472316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.473117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.474720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022864d-55b6-4e9c-abf0-bdc21befa0bd admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.487416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1328/2665] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:14.491886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.503062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff6f6a-132b-4099-acab-567f9e24ba2b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.508182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.509192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.509449 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.509774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] PATH_INFO: `/v3/projects/ea90dc89f7ed4e3890d220acafbce3d8/users/4fbd7f1b93a24e9a8a38bd4f7421fb57/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.526224 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea90dc89f7ed4e3890d220acafbce3d8, user_id=4fbd7f1b93a24e9a8a38bd4f7421fb57, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.528938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.543890 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1338/2666] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:14 2022] PUT /identity/v3/projects/ea90dc89f7ed4e3890d220acafbce3d8/users/4fbd7f1b93a24e9a8a38bd4f7421fb57/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:22:14.554663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.554955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.555300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.583162 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] MFA Rules not processed for user `4fbd7f1b93a24e9a8a38bd4f7421fb57`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:14.584404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:14.652826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1329/2667] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:14.657436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.728956 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed330ee-2a7a-4b59-8756-d620f9d76114 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.733901 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.734566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.734854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.735161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.735843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.737733 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.768565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1339/2668] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:14.772898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.785189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947192-f95d-4622-b152-55fb162fec71 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.842956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.843578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.843866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.844142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.844832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.846755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.869227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1330/2669] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:14 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:14.874982 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.889114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3996f40-caaf-4ed4-ab69-acafed1c4aa0 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.894624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.895348 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.895651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.895922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.896651 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.898257 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.911116 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1340/2670] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:14.915692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.925759 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a711f-601f-456a-8f33-2e29f71f91f5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:14.930513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:14.931518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:14.931800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:14.932082 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] PATH_INFO: `/v3/projects/3d2a22fd7833432fac6615b09490a28d/users/7aacc8e5d49c451c89048fc17295baef/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:14.949428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2a22fd7833432fac6615b09490a28d, user_id=7aacc8e5d49c451c89048fc17295baef, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:14.952159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:14.967918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1331/2671] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:14 2022] PUT /identity/v3/projects/3d2a22fd7833432fac6615b09490a28d/users/7aacc8e5d49c451c89048fc17295baef/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:22:14.972390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:14.984723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2e677a-c8f3-419d-bc86-7708a326dbc4 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:15.034435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:15.044951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:15.045636 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.046114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.046602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.047611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:15.049239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43afbe8-240a-401f-bd73-8a41673833f3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:15.059725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1341/2672] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:15.064025 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:15.074929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6402b110-9e54-48b2-9703-44039a8696f9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:15.080473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:15.081381 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.081672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.081955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] PATH_INFO: `/v3/projects/3d2a22fd7833432fac6615b09490a28d/users/7aacc8e5d49c451c89048fc17295baef/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.084028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d2a22fd7833432fac6615b09490a28d, user_id=7aacc8e5d49c451c89048fc17295baef, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:15.086677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:15.097454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1332/2673] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:15 2022] PUT /identity/v3/projects/3d2a22fd7833432fac6615b09490a28d/users/7aacc8e5d49c451c89048fc17295baef/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:15.108063 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.108356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.108633 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.137220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] MFA Rules not processed for user `7aacc8e5d49c451c89048fc17295baef`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:15.138429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:15.209184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1342/2674] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:15.221261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:15.223488 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:15.246912 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:15.284182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515949f-eae3-4c63-83cd-c4e84511c67e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:15.288262 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:15.288924 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.289212 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.289492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.299191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:15.300672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed6089-180f-4585-b3a3-86f3c1d18138 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:15.310830 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1333/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:15 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:15.450330 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:15.519822 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7741d9-3dc7-40b7-b38c-46056241d2ef None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:15.524823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:15.526071 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.526354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.526713 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] PATH_INFO: `/v3/users/4fbd7f1b93a24e9a8a38bd4f7421fb57` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.528003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbd7f1b93a24e9a8a38bd4f7421fb57)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:15.529935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:15.579401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbd7f1b93a24e9a8a38bd4f7421fb57'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:15.594870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbd7f1b93a24e9a8a38bd4f7421fb57'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:15.602329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbd7f1b93a24e9a8a38bd4f7421fb57'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:15.603318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbd7f1b93a24e9a8a38bd4f7421fb57'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:15.603997 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbd7f1b93a24e9a8a38bd4f7421fb57'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:15.607032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:15.621123 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1343/2676] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:15 2022] DELETE /identity/v3/users/4fbd7f1b93a24e9a8a38bd4f7421fb57 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:15.625704 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.626210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.626210 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.646185 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:15.647621 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:15.711461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1334/2677] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:15.721638 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:15.723535 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:15.783275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfc0ec-2098-4166-bccb-85f075d7cbbc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:15.787042 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:15.787724 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:15.787994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:15.788302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:15.798017 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:15.799476 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:15.809424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1344/2678] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:16.096476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:16.097996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:16.111378 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cd0c5-7827-42d3-b6c4-c82714f4dadd None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:16.116938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:16.118019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:16.119153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:16.119484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] PATH_INFO: `/v3/projects/ea90dc89f7ed4e3890d220acafbce3d8` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:16.120742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea90dc89f7ed4e3890d220acafbce3d8)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:16.122680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:16.175330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea90dc89f7ed4e3890d220acafbce3d8', 'request_id': 'req-d22f7950-0625-4f6a-a590-a2f14918f1eb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.183625 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea90dc89f7ed4e3890d220acafbce3d8', 'request_id': 'req-d22f7950-0625-4f6a-a590-a2f14918f1eb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.189312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea90dc89f7ed4e3890d220acafbce3d8', 'request_id': 'req-d22f7950-0625-4f6a-a590-a2f14918f1eb'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.200723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] The token cache is being invalidate because project ea90dc89f7ed4e3890d220acafbce3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:16.201014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.201655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.203331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1335/2679] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:16 2022] DELETE /identity/v3/projects/ea90dc89f7ed4e3890d220acafbce3d8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:16.207470 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:16.217552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1834f-b104-4bfb-a9d5-be3950b4a3bf service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:16.279368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:16.280277 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:16.280563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:16.280844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] PATH_INFO: `/v3/users/7aacc8e5d49c451c89048fc17295baef` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:16.282151 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aacc8e5d49c451c89048fc17295baef)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:16.284191 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:16.337855 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aacc8e5d49c451c89048fc17295baef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.350366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aacc8e5d49c451c89048fc17295baef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.358213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aacc8e5d49c451c89048fc17295baef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.359085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aacc8e5d49c451c89048fc17295baef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.359888 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-e6fcd187-d4d5-4915-bfb3-4a489f5115ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aacc8e5d49c451c89048fc17295baef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.374883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1345/2680] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:16 2022] DELETE /identity/v3/users/7aacc8e5d49c451c89048fc17295baef => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:16.674962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:16.677100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:16.691566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22f7950-0625-4f6a-a590-a2f14918f1eb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:16.739630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:16.754798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:16.755712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:16.755994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:16.756267 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] PATH_INFO: `/v3/projects/3d2a22fd7833432fac6615b09490a28d` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:16.757432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d2a22fd7833432fac6615b09490a28d)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:16.759962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:16.819581 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d2a22fd7833432fac6615b09490a28d', 'request_id': 'req-dc9184a8-b952-464f-8004-5e9cefcfc45c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.830860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d2a22fd7833432fac6615b09490a28d', 'request_id': 'req-dc9184a8-b952-464f-8004-5e9cefcfc45c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.839186 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d2a22fd7833432fac6615b09490a28d', 'request_id': 'req-dc9184a8-b952-464f-8004-5e9cefcfc45c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.851797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] The token cache is being invalidate because project 3d2a22fd7833432fac6615b09490a28d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:16.852086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.853081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-dc9184a8-b952-464f-8004-5e9cefcfc45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:16.854746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1336/2681] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:16 2022] DELETE /identity/v3/projects/3d2a22fd7833432fac6615b09490a28d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:16.860740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:16.861361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:16.861361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:16.880832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:16.882061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:16.893032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:16.944945 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1346/2682] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:22:16.953619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:16.954108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:16.954782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:16.977424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:16.978770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:16.987852 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1337/2683] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:22:16.992278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.002664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3137b1-6b27-4653-a056-4eb51a58a31a None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.008231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.008995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.009282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.009530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.010285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:17.010595 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.012241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.023840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1347/2684] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:17.028510 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.038308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466f9fa-afdb-4e67-8f62-f83dc88ac972 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.044167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.044867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.045163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.045433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.046240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.048244 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.076245 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1338/2685] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:17.080352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.090654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8304c-606b-40df-b167-cff58255a9fb admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.149584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.150700 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.150989 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.151311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.152024 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.153970 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.177587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1348/2686] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:17.184058 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.199331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f7f5c-076b-4bbd-9698-bd7bc6fa591d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.204493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.205168 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.205459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.205744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.206477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.208206 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.220109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1339/2687] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:17.224189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.234452 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e5b3-8d31-4d1e-b139-eff613bfe43c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.239359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.240021 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.240305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.240580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.241305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.242951 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070deb0-2d04-43de-900f-a56990c3a047 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.255231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1349/2688] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:17.262361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.277706 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b7fdfd-90ab-40ef-b803-9347bf30588b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.285436 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.286631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.286938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.287288 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] PATH_INFO: `/v3/projects/c728c10cb75344619a4d7fd989b781f0/users/d58e968581354a8dac5e1c2e75dc6995/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.307958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c728c10cb75344619a4d7fd989b781f0, user_id=d58e968581354a8dac5e1c2e75dc6995, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.310714 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.326136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1340/2689] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:17 2022] PUT /identity/v3/projects/c728c10cb75344619a4d7fd989b781f0/users/d58e968581354a8dac5e1c2e75dc6995/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:17.336644 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.337155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.337900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.364190 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] MFA Rules not processed for user `d58e968581354a8dac5e1c2e75dc6995`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:17.365403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:17.438531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1350/2690] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:17.443277 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.508359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ac124-a0d4-4a37-b665-2fa219feca06 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.513970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.514640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.514927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.515242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.515937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.517860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.538600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:17.549978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1341/2691] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:17.551086 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.562271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3780bf7-2a6c-4ada-8120-20e7c9b6e66b None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.570425 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:17.620497 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.621179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.621471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.621748 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.622437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.624398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.646293 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1351/2692] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:22:17.651388 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.661791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358bd73-eb1b-41ce-aa13-65d25670608d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.667056 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.667761 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.668044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.668348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.669070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.670662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.681729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1342/2693] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:17.687505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.701058 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390cc35-6fb7-4e35-84cd-4976ef97f2c7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.706709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.707672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.707954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.708233 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] PATH_INFO: `/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101/users/d6b3a87c628a4643a0b4737025305350/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.724065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc88f9e8710f49f493f8cfc99cbc0101, user_id=d6b3a87c628a4643a0b4737025305350, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.727263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.742079 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1352/2694] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:17 2022] PUT /identity/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101/users/d6b3a87c628a4643a0b4737025305350/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:17.746419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.756853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74140e1a-30c0-4d2a-9168-7361794d12bc admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.816366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.816959 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.817241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.817511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.818234 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.819876 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4f33b-76df-40ed-9cd4-af1d4ea1e55b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.830019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1343/2695] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:17.834244 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.844719 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b60a-0a12-4a71-903c-1c5e29ddcf52 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:17.849702 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:17.850592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.850871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.851128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] PATH_INFO: `/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101/users/d6b3a87c628a4643a0b4737025305350/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.853520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc88f9e8710f49f493f8cfc99cbc0101, user_id=d6b3a87c628a4643a0b4737025305350, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:17.856177 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:17.866489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1353/2696] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:17 2022] PUT /identity/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101/users/d6b3a87c628a4643a0b4737025305350/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:17.877747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:17.878043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:17.878327 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:17.906898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] MFA Rules not processed for user `d6b3a87c628a4643a0b4737025305350`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:17.908173 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:17.979129 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1344/2697] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:17.990427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:17.992336 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:18.054133 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16009-f146-4222-830e-16ee0cd7bb38 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:18.058647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:18.059318 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:18.059606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:18.059882 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:18.118782 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:18.120165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:18.129139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1354/2698] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:22.815505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:22.826458 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbecca9-7eed-4a3f-943f-de6b64229856 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:22.831125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:22.831775 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:22.832059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:22.832331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:22.842552 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:22.844029 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143bd597-b16e-4f65-9c9c-e82c721030b5 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:22.853935 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1345/2699] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:28.125491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:32.849860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=75 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:43.837984 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:43.904212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56898c42-f19f-46d8-88a7-756bb5e891a0 service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:43.909327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:43.910213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:43.910609 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:43.911145 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] PATH_INFO: `/v3/users/d58e968581354a8dac5e1c2e75dc6995` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:43.912158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58e968581354a8dac5e1c2e75dc6995)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:43.914139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:43.962587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58e968581354a8dac5e1c2e75dc6995'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:43.974119 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58e968581354a8dac5e1c2e75dc6995'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:43.980755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58e968581354a8dac5e1c2e75dc6995'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:43.981407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58e968581354a8dac5e1c2e75dc6995'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:43.982018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58e968581354a8dac5e1c2e75dc6995'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:43.997143 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1355/2700] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:43 2022] DELETE /identity/v3/users/d58e968581354a8dac5e1c2e75dc6995 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:44.001889 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:44.002412 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:44.002523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:44.020089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:44.021334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:44.086500 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1346/2701] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:44.096562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:44.098101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:44.158942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1e5d-79a1-41d0-9633-662cea7383dd admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:44.163124 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:44.163828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:44.164117 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:44.164394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:44.173678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:44.175131 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:44.184827 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1356/2702] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:44.509353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:44.510920 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:44.524707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacf960-826a-4bba-82a0-cdf975fd2260 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:44.530276 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:44.531189 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:44.531491 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:44.531770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] PATH_INFO: `/v3/projects/c728c10cb75344619a4d7fd989b781f0` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:44.533002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c728c10cb75344619a4d7fd989b781f0)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:44.534966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:44.585624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c728c10cb75344619a4d7fd989b781f0', 'request_id': 'req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.595219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c728c10cb75344619a4d7fd989b781f0', 'request_id': 'req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.601408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c728c10cb75344619a4d7fd989b781f0', 'request_id': 'req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.612266 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] The token cache is being invalidate because project c728c10cb75344619a4d7fd989b781f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:44.612553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.613191 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.614785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1347/2703] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:44 2022] DELETE /identity/v3/projects/c728c10cb75344619a4d7fd989b781f0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:44.618886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:44.628793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7eee0-6ad7-4550-b871-e51366b9e971 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:44.648302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:44.687112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:44.688039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:44.688326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:44.688606 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] PATH_INFO: `/v3/users/d6b3a87c628a4643a0b4737025305350` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:44.689809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6b3a87c628a4643a0b4737025305350)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:44.691774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:44.738931 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6b3a87c628a4643a0b4737025305350'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.760420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6b3a87c628a4643a0b4737025305350'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.767844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6b3a87c628a4643a0b4737025305350'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.768588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6b3a87c628a4643a0b4737025305350'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.769206 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-5f655ada-1632-42e1-a39a-670e349cd11a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6b3a87c628a4643a0b4737025305350'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:44.783812 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1357/2704] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:44 2022] DELETE /identity/v3/users/d6b3a87c628a4643a0b4737025305350 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:45.071971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.073518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:45.091269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b1d8-ead2-45ab-bc7f-6b62aea210b8 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.152856 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.153651 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.153933 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.154229 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] PATH_INFO: `/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.155495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc88f9e8710f49f493f8cfc99cbc0101)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.157437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.190574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:45.209858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc88f9e8710f49f493f8cfc99cbc0101', 'request_id': 'req-55443304-dda3-4a79-b3ee-391a2c97fa49'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:45.219472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc88f9e8710f49f493f8cfc99cbc0101', 'request_id': 'req-55443304-dda3-4a79-b3ee-391a2c97fa49'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:45.227545 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc88f9e8710f49f493f8cfc99cbc0101', 'request_id': 'req-55443304-dda3-4a79-b3ee-391a2c97fa49'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:45.238568 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] The token cache is being invalidate because project fc88f9e8710f49f493f8cfc99cbc0101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:45.238857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:45.239570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-55443304-dda3-4a79-b3ee-391a2c97fa49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:45.241418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1348/2705] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:45 2022] DELETE /identity/v3/projects/fc88f9e8710f49f493f8cfc99cbc0101 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:45.246312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.246817 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.246925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.266694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:45.267980 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:45.330913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1358/2706] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:22:45.338927 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f040-ca34-4965-af93-376a25447027 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.339254 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f040-ca34-4965-af93-376a25447027 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.339536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054f040-ca34-4965-af93-376a25447027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.360334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-2054f040-ca34-4965-af93-376a25447027 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:45.361596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2054f040-ca34-4965-af93-376a25447027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:45.370157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1349/2707] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:45.374422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.384892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3de14a-376e-4ad0-b4d1-bd063c30e347 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.389985 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.390649 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.390938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.391263 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.391999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:45.392280 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.393940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.409569 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1359/2708] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:45.414225 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054f040-ca34-4965-af93-376a25447027 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.424079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054f040-ca34-4965-af93-376a25447027 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.429957 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.430618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.430901 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.431217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.431907 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.433808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.464661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1350/2709] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:45.469305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.484156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c18ee5-25f2-49ad-8b47-ddcf678bfbb6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.520331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:45.555064 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.555795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.556075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.556345 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.557035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.558943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.579738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1360/2710] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:45.583700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.594791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30e733-3765-4f0c-8078-d5232d8e7937 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.600588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.601274 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.601559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.601837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.602578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.604237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.615528 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1351/2711] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:45.620995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.631909 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c573205b-1706-4056-8f9d-29772fb7a6c2 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.637996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.638671 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.638950 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.639268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.640030 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.641665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92e02b-ab6c-4aca-9c7b-e7c4b49e530a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.652981 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1361/2712] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:45.657362 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.668114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fd301-e27e-4036-845c-5234e73eaf4b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.673171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.674059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.674338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.674614 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] PATH_INFO: `/v3/projects/72cd012fb5334339800e6c93deba1a2e/users/a83dd3d4a81a48c6b3cb11aaa03a3c47/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.689074 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] RBAC: Authorizing `identity:create_grant(project_id=72cd012fb5334339800e6c93deba1a2e, user_id=a83dd3d4a81a48c6b3cb11aaa03a3c47, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.691753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.706999 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1352/2713] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:45 2022] PUT /identity/v3/projects/72cd012fb5334339800e6c93deba1a2e/users/a83dd3d4a81a48c6b3cb11aaa03a3c47/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:45.717689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.718001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.718285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.745666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] MFA Rules not processed for user `a83dd3d4a81a48c6b3cb11aaa03a3c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:45.746887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:45.814775 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1362/2714] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:45.827943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.893431 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab631d4-e421-45f1-9067-e2e64f3040da admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:45.899735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:45.900416 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:45.900700 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:45.900976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:45.901673 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:45.903620 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:45.937106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1353/2715] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:45.941441 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:45.953332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a968a84-c210-44c7-b5b7-6a1c32699d0f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.015057 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.015772 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.016062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.016347 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.017041 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.018992 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.039660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1363/2716] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:45 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:46.043798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.055347 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3f7d-6cff-4d34-b5c1-945cf15d19b6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.061289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.061962 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.062242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.062513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.063305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.064909 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.076722 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1354/2717] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:46.081043 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.092022 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c958f7-2aef-4748-a8c5-0d3be2d4db2a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.097346 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.098239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.098517 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.098788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] PATH_INFO: `/v3/projects/0b6782e5b2a1448187ceba788f4352c8/users/ea29b1b87dba428b8d937df4628ad83f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.114797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6782e5b2a1448187ceba788f4352c8, user_id=ea29b1b87dba428b8d937df4628ad83f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.117591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.133410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1364/2718] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:46 2022] PUT /identity/v3/projects/0b6782e5b2a1448187ceba788f4352c8/users/ea29b1b87dba428b8d937df4628ad83f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:46.137428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.147663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64615398-f5fd-49aa-851e-107f86769f00 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.168561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:46.216055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.216770 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.217039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.217312 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.218071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.219724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4940c-20f1-439e-aae0-c6a482514c05 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.229872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1355/2719] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:46.234100 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.239955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:46.245979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602f4e-4dfc-4659-90d2-d96d0674353b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.252217 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.253120 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.253402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.253678 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] PATH_INFO: `/v3/projects/0b6782e5b2a1448187ceba788f4352c8/users/ea29b1b87dba428b8d937df4628ad83f/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.255937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6782e5b2a1448187ceba788f4352c8, user_id=ea29b1b87dba428b8d937df4628ad83f, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.258578 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.269734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1365/2720] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:46 2022] PUT /identity/v3/projects/0b6782e5b2a1448187ceba788f4352c8/users/ea29b1b87dba428b8d937df4628ad83f/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:46.281612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.281908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.282198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.308870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] MFA Rules not processed for user `ea29b1b87dba428b8d937df4628ad83f`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:46.310072 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:46.381505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1356/2721] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:46.393971 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.395955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:46.454474 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669d1b9-54c4-45ee-ace7-6a161590255c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.458538 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.459182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.459493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.459768 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.469155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.470531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.480239 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1366/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:46.835464 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.845818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00c178-906a-42e1-bc73-a91f5ed81437 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.851723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29b1b87dba428b8d937df4628ad83f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b6782e5b2a1448187ceba788f4352c8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.852394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.852691 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.853054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.853809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:46.854126 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.855777 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.868514 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1357/2723] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:46.874602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.884341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b353826-cab2-4f96-81c0-ef61ffd339fd service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:46.889773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29b1b87dba428b8d937df4628ad83f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b6782e5b2a1448187ceba788f4352c8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:46.890436 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:46.890747 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:46.891028 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:46.891756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:46.893670 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:46.923427 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1367/2724] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:46 2022] POST /identity/v3/projects => generated 353 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:46.929010 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:46.944989 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9340f-9f75-4a0b-ad96-8cd07d3377e7 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:47.006399 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29b1b87dba428b8d937df4628ad83f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b6782e5b2a1448187ceba788f4352c8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:47.006994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:47.007314 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:47.007597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:47.008330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:47.008638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:47.010258 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:47.025477 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1358/2725] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:47.062828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:47.074092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758be53-238d-428f-8169-880bdbe1e017 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:47.079699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea29b1b87dba428b8d937df4628ad83f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b6782e5b2a1448187ceba788f4352c8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:47.080582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:47.080877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:47.081162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] PATH_INFO: `/v3/projects/83aab90bb70340788ebb3f71dd3a3008` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:47.082499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorizing `identity:delete_project(project_id=83aab90bb70340788ebb3f71dd3a3008)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:47.084561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:47.135695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83aab90bb70340788ebb3f71dd3a3008', 'request_id': 'req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.144608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83aab90bb70340788ebb3f71dd3a3008', 'request_id': 'req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.151010 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83aab90bb70340788ebb3f71dd3a3008', 'request_id': 'req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.162034 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] The token cache is being invalidate because project 83aab90bb70340788ebb3f71dd3a3008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:47.162326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.162963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-97a3dd2b-409a-445c-a609-bb7f9a6c30a8 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.164573 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1368/2726] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:47 2022] DELETE /identity/v3/projects/83aab90bb70340788ebb3f71dd3a3008 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:47.373061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:47.374974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:47.406573 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:47.441547 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2de19-8879-4b6e-a337-0ed36545d5f4 tempest-VolumeQuotaClassesTest-399707397 tempest-VolumeQuotaClassesTest-399707397-project-admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:47.500379 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:47.501272 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:47.501561 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:47.501840 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] PATH_INFO: `/v3/users/a83dd3d4a81a48c6b3cb11aaa03a3c47` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:47.503055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83dd3d4a81a48c6b3cb11aaa03a3c47)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:47.505022 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:47.557159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83dd3d4a81a48c6b3cb11aaa03a3c47'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.563853 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83dd3d4a81a48c6b3cb11aaa03a3c47'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.576348 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83dd3d4a81a48c6b3cb11aaa03a3c47'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.577054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83dd3d4a81a48c6b3cb11aaa03a3c47'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.585579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83dd3d4a81a48c6b3cb11aaa03a3c47'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:47.591939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1359/2727] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:47 2022] DELETE /identity/v3/users/a83dd3d4a81a48c6b3cb11aaa03a3c47 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:47.596557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:47.597045 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:47.597146 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:47.615816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:47.617032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:47.667437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:47.681198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1369/2728] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:47.692880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:47.695196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:47.752579 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ebd95-e64d-4aca-9af8-367175170e1e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:47.756729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:47.757390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:47.757674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:47.757956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:47.765376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:47.768044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:47.770109 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:47.781217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1360/2729] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:48.056999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:48.058519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:48.072012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8c573-4cb2-4b74-9e00-9c52b9c1e2ea None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:48.077652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:48.078568 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.078847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.079122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] PATH_INFO: `/v3/projects/72cd012fb5334339800e6c93deba1a2e` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.080341 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] RBAC: Authorizing `identity:delete_project(project_id=72cd012fb5334339800e6c93deba1a2e)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:48.082253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:48.132654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72cd012fb5334339800e6c93deba1a2e', 'request_id': 'req-989b7af3-a931-48e9-a856-a9f659b3580b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.141410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72cd012fb5334339800e6c93deba1a2e', 'request_id': 'req-989b7af3-a931-48e9-a856-a9f659b3580b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.147783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72cd012fb5334339800e6c93deba1a2e', 'request_id': 'req-989b7af3-a931-48e9-a856-a9f659b3580b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.159482 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] The token cache is being invalidate because project 72cd012fb5334339800e6c93deba1a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:48.159797 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.160440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.162060 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1370/2730] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:48 2022] DELETE /identity/v3/projects/72cd012fb5334339800e6c93deba1a2e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:48.166499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:48.176891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70432899-d02b-45a9-91d4-1588fe384151 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:48.235795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:48.236683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.236960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.237231 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] PATH_INFO: `/v3/users/ea29b1b87dba428b8d937df4628ad83f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.238433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea29b1b87dba428b8d937df4628ad83f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:48.240692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:48.288481 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea29b1b87dba428b8d937df4628ad83f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.295630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea29b1b87dba428b8d937df4628ad83f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.312334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea29b1b87dba428b8d937df4628ad83f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.313394 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea29b1b87dba428b8d937df4628ad83f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.326677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d2d92439-e69d-42e7-8b46-dd7936678b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea29b1b87dba428b8d937df4628ad83f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.332917 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1361/2731] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:48 2022] DELETE /identity/v3/users/ea29b1b87dba428b8d937df4628ad83f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:48.616435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:48.617916 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:48.631219 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b7af3-a931-48e9-a856-a9f659b3580b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:48.688801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:48.689631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.689895 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.690174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] PATH_INFO: `/v3/projects/0b6782e5b2a1448187ceba788f4352c8` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.691445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6782e5b2a1448187ceba788f4352c8)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:48.693382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:48.745103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6782e5b2a1448187ceba788f4352c8', 'request_id': 'req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.755332 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6782e5b2a1448187ceba788f4352c8', 'request_id': 'req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.762499 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6782e5b2a1448187ceba788f4352c8', 'request_id': 'req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.778272 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] The token cache is being invalidate because project 0b6782e5b2a1448187ceba788f4352c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:48.778566 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.779352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-322508a3-9d93-4549-b6cb-4bcf4c51e3b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:48.781026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1371/2732] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:48 2022] DELETE /identity/v3/projects/0b6782e5b2a1448187ceba788f4352c8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:48.786108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.786596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.786697 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.805386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:48.806608 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:48.870602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1362/2733] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:48.879496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.879900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.880291 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.893496 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:48.900503 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:48.902243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:48.911680 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1372/2734] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:48.916013 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:48.926125 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab59f5-d9f2-4b4c-957b-680294500c64 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:48.931085 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:48.931767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.932051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.932326 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.933059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:48.933366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:48.935070 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:48.947199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1363/2735] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:48.951428 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:48.961498 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150745e-6f3f-4755-b024-4b450a994fb2 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:48.967258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:48.967908 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:48.968192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:48.968491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:48.969157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:48.971051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.001059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1373/2736] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:48 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:49.004663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.010226 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:49.016282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ae023-3c33-4ad6-bc69-edb6cc4afc3b admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.075739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.076425 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.076713 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.076991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.077686 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.079654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.100136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1364/2737] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:49 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:22:49.104463 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.115200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cf209-4463-4305-b286-ed00be7b3750 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.121359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.122032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.122320 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.122599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.123378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.125002 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.135870 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1374/2738] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:49.140344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.155371 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c564ba7-270c-40df-a1e3-dfe398725b87 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.164083 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.164767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.165055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.165331 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.166068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.167705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e31d6-a16f-4087-a2ba-b6a41772cc89 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.178243 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1365/2739] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:49.183174 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.198751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66768264-750f-449f-a0a1-ae7115215ddb admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.205365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.206243 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.206523 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.206801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] PATH_INFO: `/v3/projects/78d3499ff10a413db30395e7f93e9f4a/users/817df2c39391450db2bcc07017c51f2c/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.222592 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d3499ff10a413db30395e7f93e9f4a, user_id=817df2c39391450db2bcc07017c51f2c, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.225329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.240694 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1375/2740] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:49 2022] PUT /identity/v3/projects/78d3499ff10a413db30395e7f93e9f4a/users/817df2c39391450db2bcc07017c51f2c/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:22:49.252414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.252834 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.253117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.281702 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] MFA Rules not processed for user `817df2c39391450db2bcc07017c51f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:49.282930 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:49.353872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1366/2741] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:49.369588 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.371550 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:49.430020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ab4f3-62a5-4122-8995-994339a9f5f1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.434029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.434674 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.434960 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.435268 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.443891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.445358 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315efdce-c9e8-46f1-98d4-57ef99982c0e service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.455352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1376/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:49.477747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.545774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b6f1-aad6-4ebf-ab5c-787d69e8bce1 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.551918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.552815 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.553093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.553373 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] PATH_INFO: `/v3/users/817df2c39391450db2bcc07017c51f2c` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.554696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] RBAC: Authorizing `identity:delete_user(user_id=817df2c39391450db2bcc07017c51f2c)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.556656 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.569780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:49.610861 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817df2c39391450db2bcc07017c51f2c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:49.618956 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817df2c39391450db2bcc07017c51f2c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:49.632103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817df2c39391450db2bcc07017c51f2c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:49.632969 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817df2c39391450db2bcc07017c51f2c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:49.643053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817df2c39391450db2bcc07017c51f2c'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:49.650338 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1367/2743] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:22:49 2022] DELETE /identity/v3/users/817df2c39391450db2bcc07017c51f2c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:49.655922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.656735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.656735 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.676306 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:49.677627 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:49.737420 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1377/2744] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:49.747960 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:49.750508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:49.815454 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699bac3-e043-4b7a-aba7-be5f2958f30e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:49.819668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:49.820417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:49.820698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:49.820974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:49.830343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:49.831810 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f48380-6af0-4faa-b484-14953ae3f5d5 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:49.841387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1368/2745] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:22:50.131969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.133773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:50.147475 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2059944-581b-4bbe-8230-7e8dbf52cf32 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.153073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.154103 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.154640 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.154940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] PATH_INFO: `/v3/projects/78d3499ff10a413db30395e7f93e9f4a` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.156362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d3499ff10a413db30395e7f93e9f4a)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.158282 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.203869 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:50.211404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d3499ff10a413db30395e7f93e9f4a', 'request_id': 'req-2e11c117-c391-48be-b0ba-9129c6347a3b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:50.219995 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d3499ff10a413db30395e7f93e9f4a', 'request_id': 'req-2e11c117-c391-48be-b0ba-9129c6347a3b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:50.225930 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d3499ff10a413db30395e7f93e9f4a', 'request_id': 'req-2e11c117-c391-48be-b0ba-9129c6347a3b'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:50.237327 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] The token cache is being invalidate because project 78d3499ff10a413db30395e7f93e9f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:22:50.237624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:50.238285 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-2e11c117-c391-48be-b0ba-9129c6347a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:22:50.239887 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1378/2746] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:22:50 2022] DELETE /identity/v3/projects/78d3499ff10a413db30395e7f93e9f4a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:22:50.247078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.247342 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.247692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.267938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:50.269152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:50.331117 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1369/2747] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:50.339118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.339424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.339734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.358422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:50.359665 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:50.367972 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1379/2748] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:50.372782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.386971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ccd02-9329-40f8-9147-3ae821d64bc3 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.393217 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.393888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.394158 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.394432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.395199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:22:50.395523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.397152 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.410160 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1370/2749] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:22:50.416326 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.426352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af25d-7493-45ea-8f2e-484c0f30b08f None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.431362 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.432018 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.432305 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.432580 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.433271 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.435209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.463407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1380/2750] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:22:50 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:22:50.467590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.478116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af849b5-f096-4639-9f42-b19c1e963bd6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.495773 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:22:50.537605 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.538260 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.538541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.538892 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.539609 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.541574 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.566460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1371/2751] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:22:50 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:22:50.571552 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.583624 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a7f07-a67a-40ff-bed8-53b3a4beaa71 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.588572 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.589252 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.589539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.589821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.590561 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.592218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.603510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1381/2752] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:50.607707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.619478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df68e7-7b20-441e-aec2-62927013d2a3 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.625680 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.626439 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.626746 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.627081 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.627976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.629618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9d7ab-3e6f-45d5-b0ab-999bdb1d41e5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.640387 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1372/2753] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:22:50.644515 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.654954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2b49a-7bb2-46e5-90bc-6fe98f3a4311 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.660563 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.661443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.661740 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.662013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] PATH_INFO: `/v3/projects/d9d1bbe5e19545e7a304381c29edc020/users/f13ccd0eb60d4f33983c3cfa17d2a451/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.679004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d1bbe5e19545e7a304381c29edc020, user_id=f13ccd0eb60d4f33983c3cfa17d2a451, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.681767 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.698691 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1382/2754] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:22:50 2022] PUT /identity/v3/projects/d9d1bbe5e19545e7a304381c29edc020/users/f13ccd0eb60d4f33983c3cfa17d2a451/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:22:50.710050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.710417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.710716 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.740511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] MFA Rules not processed for user `f13ccd0eb60d4f33983c3cfa17d2a451`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:22:50.741725 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:50.815060 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1373/2755] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:22:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:22:50.826389 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:22:50.828429 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:22:50.888695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de92b8c-635a-4341-8f8b-5b5fb876cfd6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:22:50.893256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:22:50.893905 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:22:50.894188 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:22:50.894462 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:22:50.903871 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:22:50.905361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6afca-3b20-4659-ad52-26b0eb6b8b50 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:22:50.914455 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1383/2756] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:22:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:00.811845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:00.863552 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:00.910167 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:00.930720 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c33c3a-8706-4e16-9eef-aef53eedf0fb None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:00.936756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:00.937658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:00.937942 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:00.938196 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] PATH_INFO: `/v3/users/f13ccd0eb60d4f33983c3cfa17d2a451` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:00.939672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13ccd0eb60d4f33983c3cfa17d2a451)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:00.941683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:00.996655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13ccd0eb60d4f33983c3cfa17d2a451'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.003735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13ccd0eb60d4f33983c3cfa17d2a451'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.015723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13ccd0eb60d4f33983c3cfa17d2a451'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.016453 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13ccd0eb60d4f33983c3cfa17d2a451'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.025884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13ccd0eb60d4f33983c3cfa17d2a451'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.033724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1374/2757] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:23:00 2022] DELETE /identity/v3/users/f13ccd0eb60d4f33983c3cfa17d2a451 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:23:01.039959 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:01.040278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:01.040553 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:01.062200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:23:01.063494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:01.127001 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1384/2758] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:23:01.137050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:01.138628 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:01.199239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe042dd-c6d5-4cb9-ade8-65093d702ad4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:01.205546 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:01.206513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:01.206860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:01.207242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:01.221104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:01.222566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:01.232800 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1375/2759] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:01.531925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:01.533549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:01.549052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c20a1f-8b76-4213-9ab3-5c7b311533f1 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:01.553973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:01.554891 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:01.555357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:01.555444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] PATH_INFO: `/v3/projects/d9d1bbe5e19545e7a304381c29edc020` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:01.556668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d1bbe5e19545e7a304381c29edc020)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:01.558586 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:01.608059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d1bbe5e19545e7a304381c29edc020', 'request_id': 'req-1580b939-d779-4dd3-aa13-61791d5b60b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.616200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d1bbe5e19545e7a304381c29edc020', 'request_id': 'req-1580b939-d779-4dd3-aa13-61791d5b60b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.623235 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d1bbe5e19545e7a304381c29edc020', 'request_id': 'req-1580b939-d779-4dd3-aa13-61791d5b60b6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.639783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] The token cache is being invalidate because project d9d1bbe5e19545e7a304381c29edc020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:23:01.640067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.640754 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:01.642367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1385/2760] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:23:01 2022] DELETE /identity/v3/projects/d9d1bbe5e19545e7a304381c29edc020 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:23:06.907986 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:06.909661 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:06.922100 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ead46-355e-4d64-a90a-cec6891fe837 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:06.941932 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:06.976204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:06.976854 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:06.977143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:06.977417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:06.979418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:07.039755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:07.041934 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:07.053451 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1376/2761] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:11.147679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:11.149571 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:11.222611 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b939-d779-4dd3-aa13-61791d5b60b6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:11.228290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:11.228957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:11.229352 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:11.229745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:11.231616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:11.243804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:11.245464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:11.256657 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1386/2762] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:11.998096 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:11.999723 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:12.011011 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033b8f1-3b97-43ce-ab48-6c6239b8237c service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:12.015536 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:12.016192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:12.016484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:12.016774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:12.018842 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:12.081955 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:12.083438 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:12.094038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1377/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:15.027814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:15.041288 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58dde9-714a-4cc7-89f7-9c8a8861a26d service placement] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:15.047202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:15.048012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:15.048402 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:15.048780 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:15.050765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:15.064622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:15.066777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:15.074531 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:15.081176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1387/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:18.361679 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:18.363219 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:18.377018 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc855b-7154-4a99-8728-3b106c59beb5 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:18.381948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:18.382849 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:18.383166 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:18.383447 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] PATH_INFO: `/v3/users/ebfcd2137bf94726a25fea389ef24430` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:18.384707 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebfcd2137bf94726a25fea389ef24430)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:18.386857 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:18.461495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebfcd2137bf94726a25fea389ef24430'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.462375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebfcd2137bf94726a25fea389ef24430'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.472874 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:18.479316 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebfcd2137bf94726a25fea389ef24430'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.480511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebfcd2137bf94726a25fea389ef24430'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.489111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebfcd2137bf94726a25fea389ef24430'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.495789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1378/2765] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:23:18 2022] DELETE /identity/v3/users/ebfcd2137bf94726a25fea389ef24430 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:23:18.688715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:18.705317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318ddf0-055d-4336-b7f1-00347f3ba8e6 service placement] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:18.782390 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:18.783439 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:18.783777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:18.784149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] PATH_INFO: `/v3/projects/62d0dd3cae1448049dc95fb347fd1a80` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:18.785504 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d0dd3cae1448049dc95fb347fd1a80)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:18.787648 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:18.841372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d0dd3cae1448049dc95fb347fd1a80', 'request_id': 'req-9b26cddb-38f0-4bf4-af30-22e5c129df6a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.849715 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d0dd3cae1448049dc95fb347fd1a80', 'request_id': 'req-9b26cddb-38f0-4bf4-af30-22e5c129df6a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.855471 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d0dd3cae1448049dc95fb347fd1a80', 'request_id': 'req-9b26cddb-38f0-4bf4-af30-22e5c129df6a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.868400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] The token cache is being invalidate because project 62d0dd3cae1448049dc95fb347fd1a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:23:18.868663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.869399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:18.871736 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1388/2766] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:23:18 2022] DELETE /identity/v3/projects/62d0dd3cae1448049dc95fb347fd1a80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:23:18.876596 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:18.878086 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:18.891295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6767ad-a476-4937-a892-2cdd71d494c9 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:18.952685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:18.953582 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:18.953868 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:18.954177 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] PATH_INFO: `/v3/users/dc0a9730c0774266a46d297498f7d496` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:18.955415 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0a9730c0774266a46d297498f7d496)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:18.957345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:19.005197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0a9730c0774266a46d297498f7d496'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.012111 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0a9730c0774266a46d297498f7d496'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.026320 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0a9730c0774266a46d297498f7d496'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.027006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0a9730c0774266a46d297498f7d496'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.036948 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0a9730c0774266a46d297498f7d496'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.043157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1379/2767] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:23:18 2022] DELETE /identity/v3/users/dc0a9730c0774266a46d297498f7d496 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 04:23:19.231539 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:19.233357 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:19.248383 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26cddb-38f0-4bf4-af30-22e5c129df6a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:19.308279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:19.309265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:19.309618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:19.309902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] PATH_INFO: `/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:19.311105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f161ea30f4f4748a753e6fe6f28efe9)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:19.313138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:19.350385 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:19.364587 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f161ea30f4f4748a753e6fe6f28efe9', 'request_id': 'req-fbdb031a-e19f-43c0-97ca-c86a78a27487'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.372175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f161ea30f4f4748a753e6fe6f28efe9', 'request_id': 'req-fbdb031a-e19f-43c0-97ca-c86a78a27487'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.378213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f161ea30f4f4748a753e6fe6f28efe9', 'request_id': 'req-fbdb031a-e19f-43c0-97ca-c86a78a27487'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.389914 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] The token cache is being invalidate because project 4f161ea30f4f4748a753e6fe6f28efe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:23:19.390201 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.390850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-fbdb031a-e19f-43c0-97ca-c86a78a27487 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:23:19.392448 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1389/2768] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:23:19 2022] DELETE /identity/v3/projects/4f161ea30f4f4748a753e6fe6f28efe9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:23:29.039650 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=58,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:29.388467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:30.510655 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:30.513308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:30.582163 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe8cf2-cc14-46f6-83ad-2852add1ef87 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:30.649344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:30.650139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:30.650420 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:30.650698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:30.653871 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:30.716367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:30.717822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:30.727063 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1380/2769] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:23:36.589824 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead6f7-d3c3-4383-95f6-4f4e30c2397b None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:36.590391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead6f7-d3c3-4383-95f6-4f4e30c2397b None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:36.590391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deead6f7-d3c3-4383-95f6-4f4e30c2397b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:36.609738 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-deead6f7-d3c3-4383-95f6-4f4e30c2397b None None] MFA Rules not processed for user `e269c6a656ff4721911ae25657eeedca`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:23:36.611489 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-deead6f7-d3c3-4383-95f6-4f4e30c2397b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:36.618755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1390/2770] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:23:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:23:40.722974 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=68 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:46.604751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:23:52.193299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:23:52.194975 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:52.257683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d53f57-0b97-4ac3-8ea8-5afe299aa855 service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:23:52.262351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:23:52.263008 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:23:52.263325 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:23:52.263604 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:23:52.265674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:23:52.340240 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:23:52.341677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:23:52.352391 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1381/2771] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:23:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:24:02.347563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=50 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:24:55.981727 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:24:55.982329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:24:55.982329 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:24:56.000412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] MFA Rules not processed for user `a432229e175149b0bc254f19af8007fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:24:56.001622 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:24:56.060530 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1391/2772] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:24:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:25:06.053509 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=25 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:25:33.781566 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:33.791558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2763e1-b613-4d36-8fdb-8319537f0133 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:33.795976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:33.796792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:33.797061 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:33.797427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:33.799478 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:33.870806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:33.872286 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:33.883804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1382/2773] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:25:35.094040 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:35.104859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5e82a-8713-49cd-9e4f-1d4037069997 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:35.108935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:35.109712 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:35.110061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:35.110697 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:35.112816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:35.128652 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:35.129683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:35.143642 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1392/2774] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:25:40.845486 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:40.855505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2e7f7-997d-406a-9164-0f5510e274b4 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:40.860143 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:40.861076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:40.861669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:40.862600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:40.865052 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:40.876506 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:40.878334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:40.888418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1383/2775] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:25:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:25:45.135938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=6 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:25:46.324804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:46.326231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:46.337300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f346fb-597e-4d8f-86e6-b73ba99c6b7e service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:46.340957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:46.341541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:46.341826 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:46.342105 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:46.343825 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:46.407393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:46.408844 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:46.419873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1393/2776] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:25:50.885182 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=40 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:25:52.690804 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:52.692658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:52.707728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e408f-9ec8-43b4-b532-b55db47a3ff1 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:52.713284 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:52.714209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:52.714549 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:52.714831 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] PATH_INFO: `/v3/users/e269c6a656ff4721911ae25657eeedca` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:52.716069 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e269c6a656ff4721911ae25657eeedca)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:52.717994 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:52.771554 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e269c6a656ff4721911ae25657eeedca'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:52.779915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e269c6a656ff4721911ae25657eeedca'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:52.793494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e269c6a656ff4721911ae25657eeedca'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:52.794646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e269c6a656ff4721911ae25657eeedca'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:52.808123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e269c6a656ff4721911ae25657eeedca'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:52.814728 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1384/2777] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:25:52 2022] DELETE /identity/v3/users/e269c6a656ff4721911ae25657eeedca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:25:53.009486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:53.020804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c6821-81cf-48c2-b1d1-b65235e97c15 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:53.078359 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:53.079284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:53.079565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:53.079840 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] PATH_INFO: `/v3/projects/c2fe0f4b2519494ea784cea098da6be3` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:53.081029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2fe0f4b2519494ea784cea098da6be3)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:53.082939 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:53.130416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2fe0f4b2519494ea784cea098da6be3', 'request_id': 'req-122519fa-6b7c-463a-be46-e47c5c28b98c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.138072 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2fe0f4b2519494ea784cea098da6be3', 'request_id': 'req-122519fa-6b7c-463a-be46-e47c5c28b98c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.143808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2fe0f4b2519494ea784cea098da6be3', 'request_id': 'req-122519fa-6b7c-463a-be46-e47c5c28b98c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.155494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] The token cache is being invalidate because project c2fe0f4b2519494ea784cea098da6be3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:25:53.155789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.156510 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.158183 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1394/2778] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:25:53 2022] DELETE /identity/v3/projects/c2fe0f4b2519494ea784cea098da6be3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:25:53.162509 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:53.164078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:53.178295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d61a2-be02-4b02-a752-22699cd487ad admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:53.239971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:53.240877 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:53.241164 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:53.241445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] PATH_INFO: `/v3/users/a672a95ef3594d61b8cf14d7b651b2ab` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:53.242696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a672a95ef3594d61b8cf14d7b651b2ab)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:53.244677 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:53.298846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a672a95ef3594d61b8cf14d7b651b2ab'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.305796 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a672a95ef3594d61b8cf14d7b651b2ab'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.318851 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a672a95ef3594d61b8cf14d7b651b2ab'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.319598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a672a95ef3594d61b8cf14d7b651b2ab'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.321382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:25:53.329264 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a672a95ef3594d61b8cf14d7b651b2ab'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.336695 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1385/2779] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:25:53 2022] DELETE /identity/v3/users/a672a95ef3594d61b8cf14d7b651b2ab => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:25:53.517785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:25:53.519522 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:25:53.534659 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122519fa-6b7c-463a-be46-e47c5c28b98c admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:25:53.563734 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:25:53.612411 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:25:53.613513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:25:53.613886 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:25:53.614248 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] PATH_INFO: `/v3/projects/813156a0534848c4b9eb730cc1838825` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:25:53.615600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813156a0534848c4b9eb730cc1838825)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:25:53.617521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:25:53.675771 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813156a0534848c4b9eb730cc1838825', 'request_id': 'req-88b390b1-a48a-444d-b315-c4122957d112'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.685618 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813156a0534848c4b9eb730cc1838825', 'request_id': 'req-88b390b1-a48a-444d-b315-c4122957d112'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.694763 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813156a0534848c4b9eb730cc1838825', 'request_id': 'req-88b390b1-a48a-444d-b315-c4122957d112'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.705944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] The token cache is being invalidate because project 813156a0534848c4b9eb730cc1838825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:25:53.706236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.706994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:25:53.708794 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1395/2780] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:25:53 2022] DELETE /identity/v3/projects/813156a0534848c4b9eb730cc1838825 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:26:03.333377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:03.702910 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:08.456763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:08.458548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:08.563197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca661361-80ac-4ee4-9f08-7a6ebdca2de2 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:08.633742 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:08.634418 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:08.634705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:08.634985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:08.636866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:08.649886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:08.651534 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:08.662664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1386/2781] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:16.746295 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:16.747866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:16.820614 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b390b1-a48a-444d-b315-c4122957d112 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:16.825225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:16.826108 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:16.826391 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:16.826667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] PATH_INFO: `/v3/users/a432229e175149b0bc254f19af8007fa` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:16.827853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a432229e175149b0bc254f19af8007fa)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:16.829804 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:16.878444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a432229e175149b0bc254f19af8007fa'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:16.889937 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a432229e175149b0bc254f19af8007fa'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:16.897255 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a432229e175149b0bc254f19af8007fa'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:16.897911 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a432229e175149b0bc254f19af8007fa'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:16.898500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c4df2cef-0e80-4b42-aaea-ab12b7e0621f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a432229e175149b0bc254f19af8007fa'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:16.912308 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1396/2782] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:26:16 2022] DELETE /identity/v3/users/a432229e175149b0bc254f19af8007fa => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:26:17.088675 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.104170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feeb36c-e4b0-4928-8491-90c3dac1af9a service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.165996 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.166873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.167188 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.167463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] PATH_INFO: `/v3/projects/cec258de3eea4a12a7bd9b2b9832dce6` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.168672 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec258de3eea4a12a7bd9b2b9832dce6)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.170647 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.201726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:17.219047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec258de3eea4a12a7bd9b2b9832dce6', 'request_id': 'req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:17.228305 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec258de3eea4a12a7bd9b2b9832dce6', 'request_id': 'req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:17.234811 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec258de3eea4a12a7bd9b2b9832dce6', 'request_id': 'req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:17.246627 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] The token cache is being invalidate because project cec258de3eea4a12a7bd9b2b9832dce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:26:17.247067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:17.247755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4e6591cc-82a5-42ed-8a3f-a5cfb375bcf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:26:17.249363 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1387/2783] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:26:17 2022] DELETE /identity/v3/projects/cec258de3eea4a12a7bd9b2b9832dce6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:26:17.254494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.254799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.255088 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.275182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:26:17.276367 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:17.339457 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1397/2784] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:26:17.347685 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.347953 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.348230 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.369548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:26:17.370795 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:17.380368 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1388/2785] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:26:17.384976 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.395467 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58650c-bcff-4952-a0ed-b99e76cd0a17 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.401723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.402473 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.402821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.403202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.404004 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:26:17.404388 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.406068 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.418798 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1398/2786] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:26:17.424377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.436273 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edd9d8-f006-4182-bf29-d557d2b201e5 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.441145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.441829 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.442123 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.442406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.443159 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.445034 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.475971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1389/2787] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:26:17 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:26:17.477883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.488173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f640a8-7164-4a4c-bf1d-976a2d087faa admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.492265 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:17.549620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.550231 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.550521 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.550856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.551559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.553520 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.575486 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1399/2788] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:26:17 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:26:17.579696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.590618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e7bb-a780-443c-b1a4-11ecb3cecaf0 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.595719 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.596424 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.596709 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.596991 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.597753 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.599398 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.610624 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1390/2789] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:26:17.614751 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.625059 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319704f4-7274-4a6d-bf0c-84a6b388ef87 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.630404 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.631067 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.631386 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.631666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.632399 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.634127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12166cb-65f0-4fb2-9c32-07539e0b1425 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.644698 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1400/2790] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:26:17.651395 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.662039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3744c6-caac-450d-bcdf-8118814733d2 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.667782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.668669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.668985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.669261 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] PATH_INFO: `/v3/projects/21b8001d268445c0a28c3ed02b9afcb3/users/cab10b29f56246d1af122cfcbe20b2a6/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.684570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b8001d268445c0a28c3ed02b9afcb3, user_id=cab10b29f56246d1af122cfcbe20b2a6, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.687295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.703187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1391/2791] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:26:17 2022] PUT /identity/v3/projects/21b8001d268445c0a28c3ed02b9afcb3/users/cab10b29f56246d1af122cfcbe20b2a6/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:26:17.708234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d3ab4-011f-4e91-864a-4add48cbeeca None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.708562 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d3ab4-011f-4e91-864a-4add48cbeeca None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.708843 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d3ab4-011f-4e91-864a-4add48cbeeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.725806 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c09d3ab4-011f-4e91-864a-4add48cbeeca None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:26:17.727089 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c09d3ab4-011f-4e91-864a-4add48cbeeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:17.789107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1401/2792] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:26:17.799694 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:17.801556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:17.865413 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e0511-94d1-4fb9-8986-8b12f0e36d38 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:17.870982 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:17.871886 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:17.872248 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:17.872583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:17.887455 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:17.889555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:17.902593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1392/2793] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:21.829216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dce73b-ee89-463d-afcb-aac1d0605d11 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:21.829216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dce73b-ee89-463d-afcb-aac1d0605d11 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:21.829216 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dce73b-ee89-463d-afcb-aac1d0605d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:21.863234 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-66dce73b-ee89-463d-afcb-aac1d0605d11 None None] MFA Rules not processed for user `cab10b29f56246d1af122cfcbe20b2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:26:21.864442 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-66dce73b-ee89-463d-afcb-aac1d0605d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:21.937675 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1402/2794] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:26:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:26:21.952521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:21.954114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:22.012351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:22.036401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8ad5-7aa3-4580-9fe9-e8e764e2f648 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:22.042563 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:22.043282 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:22.043575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:22.043860 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:22.053470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:22.055133 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:22.066023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1393/2795] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:23.405897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:23.406376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:23.408861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:23.429759 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] MFA Rules not processed for user `cab10b29f56246d1af122cfcbe20b2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:26:23.431412 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:23.443616 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1403/2796] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:26:23.764774 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:23.766308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:23.779468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9819bc-0b27-4201-a295-3c3c2cf2ce63 service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:23.783103 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:23.783788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:23.784073 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:23.784355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:23.786432 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:23.799824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:23.801460 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:23.810767 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1394/2797] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:25.164600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:25.166799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:25.178647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:25.258264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8d0f2-291c-4231-a394-cb0fc68e0623 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:25.263795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:25.264440 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:25.264756 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:25.265029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:25.266884 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:25.280317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:25.281947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4d545-ba84-4d3f-8739-87769d62a201 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:25.292765 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1404/2798] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:32.614434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:26:32.630785 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c020fb-1812-4fb7-ae19-4ee85330cabc service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:26:32.637055 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67b4385cb5ed42d1b9488d409abd61b1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:26:32.637718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:26:32.638006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:26:32.638295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:26:32.640237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:26:32.653630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:26:32.655246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:26:32.666642 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1395/2799] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:26:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:26:35.288189 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=28 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:26:42.662088 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=22 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:27:19.472406 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:27:19.472979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:27:19.472979 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:27:19.490626 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] MFA Rules not processed for user `cab10b29f56246d1af122cfcbe20b2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:27:19.491861 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:27:19.499128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1405/2800] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:27:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:27:19.820976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:27:19.831494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b288b-1cdf-448b-8135-85d37e56774e service nova] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:27:19.835757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:27:19.836396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:27:19.836676 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:27:19.836964 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:27:19.839406 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:27:19.849241 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:27:19.850646 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a63c17-2d55-4d8b-8534-7dcdf96b6c41 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:27:19.859239 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1396/2801] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:27:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:27:29.485454 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:27:29.855782 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=6 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:28:15.192241 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:28:15.265227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f02a0b2-7292-4f74-aa74-8842899d75b7 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:28:15.270443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:28:15.271600 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:28:15.271885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:28:15.272162 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:28:15.273917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:28:15.307679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:28:15.309200 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53fc3f-b073-41e7-8cca-b43de94bcc94 service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:28:15.319685 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1406/2802] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:28:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:28:18.180814 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:28:18.181729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:28:18.181729 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:28:18.207570 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] MFA Rules not processed for user `cab10b29f56246d1af122cfcbe20b2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:28:18.208791 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:28:18.216602 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1397/2803] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:28:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:28:25.315121 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=36 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:28:28.201592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:29:13.153647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:29:13.154214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:29:13.154214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:29:13.171477 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] MFA Rules not processed for user `cab10b29f56246d1af122cfcbe20b2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:29:13.172788 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:29:13.179952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1407/2804] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:29:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:29:23.166450 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:18.128050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.129688 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:18.148494 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a935b-e154-48c0-ba75-b152fc80a924 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:18.154204 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:18.154866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.155190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.155473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.158049 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:18.170501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:18.173101 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:18.182003 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.192631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1398/2805] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:18.224969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474009a-38f3-4839-aaf2-5d7d0742ef22 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:18.229464 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:18.230380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.230666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.230947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] PATH_INFO: `/v3/users/cab10b29f56246d1af122cfcbe20b2a6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.232205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab10b29f56246d1af122cfcbe20b2a6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:18.234273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:18.286620 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab10b29f56246d1af122cfcbe20b2a6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.298952 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab10b29f56246d1af122cfcbe20b2a6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.305725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab10b29f56246d1af122cfcbe20b2a6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.306422 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab10b29f56246d1af122cfcbe20b2a6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.307027 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-62ce8db7-fe4f-4400-a319-1dfa9d523a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab10b29f56246d1af122cfcbe20b2a6'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.321532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1408/2806] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:30:18 2022] DELETE /identity/v3/users/cab10b29f56246d1af122cfcbe20b2a6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:18.521211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.532303 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21185a8-55b1-4a1a-a8e7-59169b5e41dc service placement] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:18.596374 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:18.597271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.597557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.597863 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] PATH_INFO: `/v3/projects/21b8001d268445c0a28c3ed02b9afcb3` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.599048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b8001d268445c0a28c3ed02b9afcb3)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:18.601007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:18.651873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b8001d268445c0a28c3ed02b9afcb3', 'request_id': 'req-a9454d5f-147a-415d-ae9f-63c6a4f4d909'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.660419 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b8001d268445c0a28c3ed02b9afcb3', 'request_id': 'req-a9454d5f-147a-415d-ae9f-63c6a4f4d909'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.667038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b8001d268445c0a28c3ed02b9afcb3', 'request_id': 'req-a9454d5f-147a-415d-ae9f-63c6a4f4d909'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.694139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] The token cache is being invalidate because project 21b8001d268445c0a28c3ed02b9afcb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:30:18.694402 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.695169 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-a9454d5f-147a-415d-ae9f-63c6a4f4d909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:18.698496 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1399/2807] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:30:18 2022] DELETE /identity/v3/projects/21b8001d268445c0a28c3ed02b9afcb3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:18.704266 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.704556 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.708777 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.775917 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:18.777037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:18.846213 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1409/2808] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:18.854939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.855265 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.855548 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.874042 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:18.875271 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:18.883903 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1400/2809] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:18.888225 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.898999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d9d48-e2fd-42b7-a407-52fb24b6c071 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:18.904924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:18.905589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.905880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.906221 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.906955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:30:18.907302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:18.908933 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:18.923424 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1410/2810] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:30:18.927967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.941648 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68f257-92e8-42d7-a73c-868876a8997d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:18.947687 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:18.948383 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:18.948671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:18.948943 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:18.949664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:18.951653 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:18.980734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1401/2811] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:30:18 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:18.985139 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:18.995799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c547561-d85f-4217-b2a8-68cd5dabc455 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:19.005897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:19.060918 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:19.061494 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.061783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.062062 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.062785 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:19.064943 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:19.087187 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1411/2812] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:30:18 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:30:19.092733 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:19.106752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cde0f5-ff4e-4186-a2e1-0b73f892629f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:19.112370 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:19.113047 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.113343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.113623 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.114360 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:19.116007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:19.126518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1402/2813] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:30:19.130957 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:19.144582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e517811-cb2d-4fe1-97d3-a660c6405310 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:19.151932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:19.152996 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.153435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.153865 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.155032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:19.157128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0381c67-fa58-4b74-85cd-1dcf3a823550 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:19.168114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1412/2814] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:19.172663 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:19.186696 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1276f7-7810-49d3-bdd4-12d360b93e31 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:19.192104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:19.193015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.193298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.193575 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] PATH_INFO: `/v3/projects/831b3c496602475b873b3d99828c6d20/users/3b53074a46a14cef94f657e7568aa48f/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.208936 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] RBAC: Authorizing `identity:create_grant(project_id=831b3c496602475b873b3d99828c6d20, user_id=3b53074a46a14cef94f657e7568aa48f, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:19.211684 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:19.226533 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1403/2815] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:19 2022] PUT /identity/v3/projects/831b3c496602475b873b3d99828c6d20/users/3b53074a46a14cef94f657e7568aa48f/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:19.236876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.237372 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.237505 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.264602 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] MFA Rules not processed for user `3b53074a46a14cef94f657e7568aa48f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:19.265864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:19.330211 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1413/2816] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:19.341739 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:19.343641 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:19.348319 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:19.430023 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873608a-03e7-440a-a542-ec51ced6399e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:19.434658 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aecdfa0fc8514b09b04bd7ee759d21ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:19.435382 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:19.435678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:19.435977 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:19.445600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:19.447076 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb232be-3e11-4dd2-b009-c46a2baf9550 service glance] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:19.457300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1404/2817] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:29.326686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:29.452335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=34 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:35.039061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:35.102559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc33c-2ec8-48c1-a5b9-7ea46c354086 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:35.107084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:35.108005 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.108290 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.108564 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] PATH_INFO: `/v3/users/3b53074a46a14cef94f657e7568aa48f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.109795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b53074a46a14cef94f657e7568aa48f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:35.111862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:35.162093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b53074a46a14cef94f657e7568aa48f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.177077 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b53074a46a14cef94f657e7568aa48f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.186516 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b53074a46a14cef94f657e7568aa48f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.187283 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b53074a46a14cef94f657e7568aa48f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.188107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b53074a46a14cef94f657e7568aa48f'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.203207 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1414/2818] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:30:35 2022] DELETE /identity/v3/users/3b53074a46a14cef94f657e7568aa48f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:35.207498 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.208019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.208132 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.225750 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:35.226967 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:35.286471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1405/2819] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:35.296682 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:35.298275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:35.376696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d368550-3444-4054-84a1-cd4e3ef20dd3 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:35.380899 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:35.381621 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.381969 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.382307 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.392398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:35.394161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa859ac-830a-49d2-9d89-b011d699dde3 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:35.403942 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1415/2820] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 04:30:35.693171 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:35.694766 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:35.707848 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48e9c0-dcd2-49d6-ade5-c338a429b7e4 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:35.712872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:35.713818 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.714122 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.714377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] PATH_INFO: `/v3/projects/831b3c496602475b873b3d99828c6d20` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.715659 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=831b3c496602475b873b3d99828c6d20)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:35.717674 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:35.768669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831b3c496602475b873b3d99828c6d20', 'request_id': 'req-371d720e-7aa3-452e-a85a-89044372f477'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.777882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831b3c496602475b873b3d99828c6d20', 'request_id': 'req-371d720e-7aa3-452e-a85a-89044372f477'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.783837 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831b3c496602475b873b3d99828c6d20', 'request_id': 'req-371d720e-7aa3-452e-a85a-89044372f477'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.794787 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] The token cache is being invalidate because project 831b3c496602475b873b3d99828c6d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:30:35.795077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.795824 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-371d720e-7aa3-452e-a85a-89044372f477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:35.797645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1406/2821] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:30:35 2022] DELETE /identity/v3/projects/831b3c496602475b873b3d99828c6d20 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:35.802513 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.802807 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.803114 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.809380 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:35.823927 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:35.825186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:35.887656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1416/2822] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:35.895826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.896116 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.896404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.915351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:35.916562 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:35.925359 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1407/2823] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 04:30:35.929737 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:35.940184 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132a52d-a5ef-4905-9ff2-00e87338ab77 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:35.946132 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:35.946801 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.947085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.947392 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.948126 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:30:35.948435 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:35.950056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:35.962138 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1417/2824] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:30:35.966440 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:35.976077 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d84bd5-3e45-452b-80bd-1a24783be40a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:35.981237 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:35.981900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:35.982190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:35.982473 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:35.983236 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:35.985187 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.012397 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1408/2825] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:30:35 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:36.016666 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:36.027430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c284e4-6899-4dfe-837f-4ff8eab3f3cd admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:36.096836 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:36.097542 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.097654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.097964 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.098662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:36.100832 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.122155 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1418/2826] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:30:36 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:36.127340 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:36.142323 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946addc9-992f-4cb3-93a3-742e926aa38f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:36.148041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:36.148760 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.149053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.149366 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.150102 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:36.151764 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.162517 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1409/2827] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:30:36.166434 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:36.177281 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2207d60-f0ad-4c9b-9cad-3df9cde47e24 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:36.182382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:36.183056 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.183368 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.183647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.184387 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:36.186007 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de47e5b-edf1-4cfa-8946-1fb75ec5cde7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.196877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1419/2828] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:36.201043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:36.211553 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9711c5-68d8-4610-851a-66527835980e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:36.216504 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:36.217396 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.217683 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.217965 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] PATH_INFO: `/v3/projects/22773da9c70f4c9e9b3ee28c2719b9a5/users/bf04224076844127ac2aca80a3d7f425/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.233110 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22773da9c70f4c9e9b3ee28c2719b9a5, user_id=bf04224076844127ac2aca80a3d7f425, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:36.236046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.253531 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1410/2829] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:36 2022] PUT /identity/v3/projects/22773da9c70f4c9e9b3ee28c2719b9a5/users/bf04224076844127ac2aca80a3d7f425/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:36.269628 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.269926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.270205 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.299631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] MFA Rules not processed for user `bf04224076844127ac2aca80a3d7f425`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:36.300920 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:36.364037 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:36.370363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1420/2830] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:36.382019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:36.384006 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:36.409136 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:36.469449 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca940d-3fc2-4f50-8213-d98fe7998f9d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:36.473285 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:36.473971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:36.474227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:36.474538 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:36.483299 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:36.484784 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b066a8e-1719-4d81-ad60-aea3f11dda81 service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:36.494882 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1411/2831] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:46.371672 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:46.490027 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=26 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:54.258378 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:54.325279 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4858036-55ee-4ed8-a79a-bbc00b7047e0 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:54.330366 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:54.331303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:54.331584 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:54.331867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] PATH_INFO: `/v3/users/bf04224076844127ac2aca80a3d7f425` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:54.333081 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf04224076844127ac2aca80a3d7f425)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:54.335061 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:54.385039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf04224076844127ac2aca80a3d7f425'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.397173 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf04224076844127ac2aca80a3d7f425'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.404382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf04224076844127ac2aca80a3d7f425'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.405096 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf04224076844127ac2aca80a3d7f425'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.405758 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf04224076844127ac2aca80a3d7f425'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.419880 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1421/2832] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:30:54 2022] DELETE /identity/v3/users/bf04224076844127ac2aca80a3d7f425 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:54.424660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:54.425149 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:54.425341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:54.443390 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:54.444593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:54.505192 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1412/2833] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:54.515328 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:54.516893 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:54.577363 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f63d8-a8fc-4708-8803-339fe334925d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:54.581859 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:54.582526 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:54.582809 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:54.583087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:54.592749 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:54.594215 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e99482-c98c-4297-99a5-96c4d1e43ab5 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:54.604029 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1422/2834] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:54.903619 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:54.905165 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:54.918253 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f4fdb-a845-4fd1-87e1-e7c09e78117b None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:54.924144 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:54.925020 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:54.925301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:54.925578 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] PATH_INFO: `/v3/projects/22773da9c70f4c9e9b3ee28c2719b9a5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:54.926898 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22773da9c70f4c9e9b3ee28c2719b9a5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:54.928884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:54.978592 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22773da9c70f4c9e9b3ee28c2719b9a5', 'request_id': 'req-636fca4a-3aa5-4770-94c7-62b3aef986b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.986048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22773da9c70f4c9e9b3ee28c2719b9a5', 'request_id': 'req-636fca4a-3aa5-4770-94c7-62b3aef986b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:54.991618 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22773da9c70f4c9e9b3ee28c2719b9a5', 'request_id': 'req-636fca4a-3aa5-4770-94c7-62b3aef986b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:55.003631 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] The token cache is being invalidate because project 22773da9c70f4c9e9b3ee28c2719b9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:30:55.003929 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:55.004643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-636fca4a-3aa5-4770-94c7-62b3aef986b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:30:55.006529 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1413/2835] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:30:54 2022] DELETE /identity/v3/projects/22773da9c70f4c9e9b3ee28c2719b9a5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:55.011613 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.011912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.012194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.031987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:55.033214 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:55.051559 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:55.096317 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1423/2836] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:55.104788 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.105269 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.105826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.129183 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:55.130404 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:55.139497 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1414/2837] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:55.144274 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.154264 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0621d6-f4ef-466c-8882-51df44f13982 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.158963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.159664 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.159954 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.160236 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.160973 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80702) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:30:55.161289 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.162919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.175209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1424/2838] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:30:55.179501 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.188978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1d1f7-e3ff-4e16-b6ef-eb250e79379a None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.194427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.195089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.195409 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.195690 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.196386 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.198565 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.228866 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1415/2839] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:55.232828 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.245381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e64f9a-352d-480a-8d9c-11371eee8ff8 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.311331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.312147 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.312261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.312558 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.313261 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.315276 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.339854 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1425/2840] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:30:55.344030 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.355001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1904a-0cc5-44c2-9bac-488ce9c93f59 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.360405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.361079 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.361468 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.361780 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.362516 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.364407 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.375689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1416/2841] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:55.379934 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.390924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485c039-cb47-45f3-af35-460be13166c6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.396408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.397085 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.397373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.397654 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.398397 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.400220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2870a-2e9e-4549-a9ef-57227e098c8e admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.410557 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1426/2842] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:55.414630 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.425952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca2d651-3b92-42eb-a0dd-27ec374d6fdf admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.431789 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.432872 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.433352 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.433640 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] PATH_INFO: `/v3/projects/3b759ce6c0fe4a668ed8efa997e2f04f/users/f36915d6e5334908ab0d25dc07b2201f/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.449958 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b759ce6c0fe4a668ed8efa997e2f04f, user_id=f36915d6e5334908ab0d25dc07b2201f, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.452692 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.467401 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1417/2843] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:55 2022] PUT /identity/v3/projects/3b759ce6c0fe4a668ed8efa997e2f04f/users/f36915d6e5334908ab0d25dc07b2201f/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:55.479193 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.479500 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.479778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.507365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] MFA Rules not processed for user `f36915d6e5334908ab0d25dc07b2201f`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:55.508582 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:55.595716 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1427/2844] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:55.601002 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.624476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:55.672107 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc038765-56f9-461b-b924-562c4cde2e90 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.677351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.678015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.678300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.678580 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.679300 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.681335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.708001 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1418/2845] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:55.715419 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.731897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87804822-fd2a-4b71-bd21-71f1adf1185e None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.744944 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:55.795773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.796492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.796783 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.797065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.797766 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.799744 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.820596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1428/2846] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:30:55.825044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.836645 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96fc4d-8060-408d-a195-3950c7c6512d admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.842705 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.843423 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.843711 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.843988 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.844752 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.846351 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.859976 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1419/2847] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:55.864679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.875773 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca38cb9-809c-4b61-b890-9aa6da7eb900 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.880645 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.881311 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.881591 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.881868 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.882599 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.884415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9250d36c-414f-4ebf-a1a5-005b7e73c7ab admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.894990 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1429/2848] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:55.899246 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:55.910434 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870d336-eb7d-46c9-840b-9dd4d0c24f13 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:55.915465 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:55.916557 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.916900 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.917193 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] PATH_INFO: `/v3/projects/6d68cd67f80346279d41141ab7cb2a94/users/cd233bc53068413180b8587c1b2ab3b2/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.933046 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d68cd67f80346279d41141ab7cb2a94, user_id=cd233bc53068413180b8587c1b2ab3b2, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:55.941015 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:55.955019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1420/2849] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:55 2022] PUT /identity/v3/projects/6d68cd67f80346279d41141ab7cb2a94/users/cd233bc53068413180b8587c1b2ab3b2/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:55.965795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:55.966112 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:55.966395 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:55.993128 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] MFA Rules not processed for user `cd233bc53068413180b8587c1b2ab3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:55.994273 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:56.060696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1430/2850] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:56.065310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.127850 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6aee70-0570-49c7-aa25-a3691ac0376e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.133503 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.135014 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.135437 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.135771 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.136793 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.139712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.170693 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1421/2851] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:30:56 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:30:56.174935 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.188703 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf041570-eb60-437c-acb7-8bce87752913 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.250479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.251076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.251398 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.251681 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] PATH_INFO: `/v3/users` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.252401 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.254403 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.278080 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1431/2852] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:30:56 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:30:56.282078 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.292985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e93020-708d-4eaf-b5e6-fd46c95b8c62 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.298821 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.299535 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.299826 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.300114 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.300847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.302459 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.313867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1422/2853] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:56.318202 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.329148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46ebed-88a1-4f1e-b958-efd57cfbc369 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.334156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.335053 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.335365 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.335663 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] PATH_INFO: `/v3/projects/c0d2176112b14372900d70f5463b7bf6/users/42eaeaa6d6644917be6d58809feb40bf/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.349373 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:56.353032 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d2176112b14372900d70f5463b7bf6, user_id=42eaeaa6d6644917be6d58809feb40bf, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.356194 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.370699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1432/2854] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:56 2022] PUT /identity/v3/projects/c0d2176112b14372900d70f5463b7bf6/users/42eaeaa6d6644917be6d58809feb40bf/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:56.375179 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.385361 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edefa1ec-948d-42cc-8c6c-5327f202b5b5 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.431433 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:30:56.449190 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.449873 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.450167 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.450456 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.451238 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.452970 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e85482-7db6-472b-9062-464af88a56d6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.463757 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1423/2855] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:30:56.468287 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.479913 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f7be-670b-4ca0-9a55-6780effbb348 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.485176 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.486220 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.486514 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.486799 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] PATH_INFO: `/v3/projects/c0d2176112b14372900d70f5463b7bf6/users/42eaeaa6d6644917be6d58809feb40bf/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.488975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d2176112b14372900d70f5463b7bf6, user_id=42eaeaa6d6644917be6d58809feb40bf, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.491699 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.503431 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1433/2856] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:30:56 2022] PUT /identity/v3/projects/c0d2176112b14372900d70f5463b7bf6/users/42eaeaa6d6644917be6d58809feb40bf/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:30:56.514654 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.515068 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.515408 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.544137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] MFA Rules not processed for user `42eaeaa6d6644917be6d58809feb40bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:30:56.545377 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:56.615461 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1424/2857] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:30:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:30:56.626851 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:56.628683 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:56.689084 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbece0-ae6b-4200-b1b5-25e394010f6d admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:56.692936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:56.693589 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:56.693877 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:56.694159 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:56.753977 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:56.755492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:56.778662 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1434/2858] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:58.337466 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:58.339603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:30:58.361841 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e46ef4-fb95-43e3-b51e-8be528937b7f None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:58.369021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:58.369668 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:58.369952 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:58.370227 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:58.444199 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:58.446337 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:58.459194 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1425/2859] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:30:59.011639 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:30:59.021315 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633e69c0-4fec-4861-9a49-2da9826f4f0f service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:30:59.025898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:30:59.026551 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:30:59.026839 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:30:59.027148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:30:59.036529 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:30:59.037967 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf4019-6d67-4e17-a18d-4a5fd524af69 service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:30:59.047527 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1435/2860] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:30:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:02.955341 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:03.029414 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02708c0-b85f-40bd-a1ef-fd721c0f499e service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:03.034638 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:03.035544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:03.035832 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:03.036108 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] PATH_INFO: `/v3/users/f36915d6e5334908ab0d25dc07b2201f` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:03.037367 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36915d6e5334908ab0d25dc07b2201f)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:03.039345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:03.044153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=100 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:03.091708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36915d6e5334908ab0d25dc07b2201f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.098798 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36915d6e5334908ab0d25dc07b2201f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.115139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36915d6e5334908ab0d25dc07b2201f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.116137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36915d6e5334908ab0d25dc07b2201f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.129308 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36915d6e5334908ab0d25dc07b2201f'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.137512 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1426/2861] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:02 2022] DELETE /identity/v3/users/f36915d6e5334908ab0d25dc07b2201f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:03.142405 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:03.142902 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:03.143025 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:03.160565 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:03.161769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:03.221883 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1436/2862] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:03.231887 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:03.233428 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:03.300335 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12d022-457c-416d-8f90-c2c8d52fea1e admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:03.304867 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:03.305523 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:03.305808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:03.306087 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:03.316059 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:03.317513 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:03.327463 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1427/2863] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:03.642253 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:03.644860 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:03.660381 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccda26-78d3-4e54-ac4e-8dab2628c133 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:03.665912 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:03.666853 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:03.667118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:03.667426 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] PATH_INFO: `/v3/projects/3b759ce6c0fe4a668ed8efa997e2f04f` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:03.668802 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b759ce6c0fe4a668ed8efa997e2f04f)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:03.670746 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:03.688478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:03.721608 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b759ce6c0fe4a668ed8efa997e2f04f', 'request_id': 'req-b875ff15-e20c-4d29-a611-90925fc2b00a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.729975 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b759ce6c0fe4a668ed8efa997e2f04f', 'request_id': 'req-b875ff15-e20c-4d29-a611-90925fc2b00a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.735846 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b759ce6c0fe4a668ed8efa997e2f04f', 'request_id': 'req-b875ff15-e20c-4d29-a611-90925fc2b00a'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.747127 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] The token cache is being invalidate because project 3b759ce6c0fe4a668ed8efa997e2f04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:03.747433 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.748172 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.749823 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1437/2864] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:03 2022] DELETE /identity/v3/projects/3b759ce6c0fe4a668ed8efa997e2f04f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:03.754000 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:03.764048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79484041-408e-4c0a-9421-5886ef9d1186 service neutron] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:03.827945 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:03.828971 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:03.829263 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:03.829540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] PATH_INFO: `/v3/users/cd233bc53068413180b8587c1b2ab3b2` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:03.830822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd233bc53068413180b8587c1b2ab3b2)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:03.832817 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:03.870845 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:03.895339 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd233bc53068413180b8587c1b2ab3b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.902355 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd233bc53068413180b8587c1b2ab3b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.914104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd233bc53068413180b8587c1b2ab3b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.914786 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd233bc53068413180b8587c1b2ab3b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.923170 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd233bc53068413180b8587c1b2ab3b2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:03.929664 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1428/2865] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:03 2022] DELETE /identity/v3/users/cd233bc53068413180b8587c1b2ab3b2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:04.212394 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:04.214052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:04.227774 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875ff15-e20c-4d29-a611-90925fc2b00a admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:04.289892 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:04.290800 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:04.291087 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:04.291430 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] PATH_INFO: `/v3/projects/6d68cd67f80346279d41141ab7cb2a94` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:04.292696 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d68cd67f80346279d41141ab7cb2a94)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:04.294631 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:04.341725 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d68cd67f80346279d41141ab7cb2a94', 'request_id': 'req-c3743426-1dac-4535-948c-df32939a48c1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.350354 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d68cd67f80346279d41141ab7cb2a94', 'request_id': 'req-c3743426-1dac-4535-948c-df32939a48c1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.357192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d68cd67f80346279d41141ab7cb2a94', 'request_id': 'req-c3743426-1dac-4535-948c-df32939a48c1'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.368251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] The token cache is being invalidate because project 6d68cd67f80346279d41141ab7cb2a94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:04.368540 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.369186 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.370816 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1438/2866] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:04 2022] DELETE /identity/v3/projects/6d68cd67f80346279d41141ab7cb2a94 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:04.374944 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:04.376392 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:04.389643 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab32cd-312e-441b-afd4-570e8a1e027f admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:04.454301 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:04.455298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:04.455550 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:04.455794 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] PATH_INFO: `/v3/users/42eaeaa6d6644917be6d58809feb40bf` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:04.457007 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42eaeaa6d6644917be6d58809feb40bf)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:04.459344 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:04.512819 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42eaeaa6d6644917be6d58809feb40bf'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.520137 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42eaeaa6d6644917be6d58809feb40bf'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.533492 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42eaeaa6d6644917be6d58809feb40bf'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.534242 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42eaeaa6d6644917be6d58809feb40bf'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.543041 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-cb5173b0-42c1-42d2-9967-a4824acf4421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42eaeaa6d6644917be6d58809feb40bf'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.549203 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1429/2867] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:04 2022] DELETE /identity/v3/users/42eaeaa6d6644917be6d58809feb40bf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:04.833109 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:04.834864 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:04.850020 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3743426-1dac-4535-948c-df32939a48c1 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:04.882158 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:04.914679 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:04.915597 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:04.915876 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:04.916148 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] PATH_INFO: `/v3/projects/c0d2176112b14372900d70f5463b7bf6` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:04.918198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0d2176112b14372900d70f5463b7bf6)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:04.920212 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:04.981900 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0d2176112b14372900d70f5463b7bf6', 'request_id': 'req-b7f6de13-1e83-486f-a7b3-61f83d003fef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.990051 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0d2176112b14372900d70f5463b7bf6', 'request_id': 'req-b7f6de13-1e83-486f-a7b3-61f83d003fef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:04.996033 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0d2176112b14372900d70f5463b7bf6', 'request_id': 'req-b7f6de13-1e83-486f-a7b3-61f83d003fef'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:05.007299 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] The token cache is being invalidate because project c0d2176112b14372900d70f5463b7bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80702) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:05.007596 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:05.008309 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-b7f6de13-1e83-486f-a7b3-61f83d003fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:05.010055 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1439/2868] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:04 2022] DELETE /identity/v3/projects/c0d2176112b14372900d70f5463b7bf6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:05.015278 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.015603 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.015910 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.034495 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:05.035718 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:05.061334 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:05.104275 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1430/2869] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:05.114009 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.114303 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.114583 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.133506 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:05.134709 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:05.143300 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1440/2870] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:05.147937 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.158050 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebce5-2f4a-4b0e-a973-1153617f2d2d None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.163145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.163822 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.164106 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.164385 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] PATH_INFO: `/v3/domains` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.165127 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80703) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 04:31:05.165884 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.167051 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.179600 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1431/2871] 10.222.0.13 () {58 vars in 1152 bytes} [Thu Jun 30 04:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 04:31:05.184209 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.194897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb35ddfe-4ad9-4435-9b26-274836034299 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.199922 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.200579 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.200866 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.201175 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.201867 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.203856 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.234695 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1441/2872] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:31:05.238671 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.251021 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f7e01-dad7-469a-aa20-bb91b78c5abb admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.317734 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.318445 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.318735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.319019 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.319755 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.321781 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.344376 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1432/2873] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:31:05.348641 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.359400 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2338c-56cd-484f-b08a-3012b5325782 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.364593 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.365258 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.365541 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.365814 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.366546 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.368180 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.378567 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1442/2874] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:31:05.384472 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.395518 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676743e2-32e9-4460-a6d8-90e984eacba4 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.400894 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.401559 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.401839 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.402104 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.402870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.404476 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88791e84-f895-4cd4-be4e-060210b8f106 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.414405 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1433/2875] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:31:05.418795 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.428789 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3eec9-9dc2-484a-8503-3ae6eb4a3d26 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.434115 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.434998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.435408 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.435686 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] PATH_INFO: `/v3/projects/633e27654f1447afa8eba59347024420/users/d916c4f90723401d8e5b0a93ba00ad8c/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.450988 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633e27654f1447afa8eba59347024420, user_id=d916c4f90723401d8e5b0a93ba00ad8c, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.453667 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.470323 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1443/2876] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:31:05 2022] PUT /identity/v3/projects/633e27654f1447afa8eba59347024420/users/d916c4f90723401d8e5b0a93ba00ad8c/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:05.481292 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.481590 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.481915 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.509698 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] MFA Rules not processed for user `d916c4f90723401d8e5b0a93ba00ad8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:05.511053 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:05.587985 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1434/2877] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:05.592987 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.654925 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f01de-435c-406a-a2e9-b1b5b2ca5759 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.660142 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.660808 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.661095 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.661376 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.662065 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.664039 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.667660 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:05.692349 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1444/2878] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:31:05.696858 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.708744 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b9e5ac-2e4c-4dbe-9ddf-160c740badb0 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.726417 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:05.778153 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.778743 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.779043 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.779343 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.780054 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.782009 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.810462 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1435/2879] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:31:05.811821 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.822549 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49fb12-52ab-441a-9b8d-3741000bbffc admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.830919 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.831955 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.832278 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.832581 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.833704 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.835898 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.848885 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1445/2880] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 04:31:05.853044 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.863908 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee747f-7070-4aed-9eb2-6ee47f7ec8d6 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.869809 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.870499 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.870790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.871075 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.871847 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.873471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adc2be-489c-4a3d-a100-05d9e10c10ed admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.884138 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1436/2881] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:31:05.888393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:05.903757 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c7f67-da5e-4d76-90b8-adc761ffd62b admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:05.909519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:05.910410 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.910689 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.910963 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] PATH_INFO: `/v3/projects/7feefd21eb33483bb64bf68462b4d8dc/users/040bc69a909d40b1a6becddcb0c853be/roles/370601cf296e44ea95990467138eb5e5` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:05.940344 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7feefd21eb33483bb64bf68462b4d8dc, user_id=040bc69a909d40b1a6becddcb0c853be, role_id=370601cf296e44ea95990467138eb5e5)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:05.944714 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:05.965478 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1446/2882] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:31:05 2022] PUT /identity/v3/projects/7feefd21eb33483bb64bf68462b4d8dc/users/040bc69a909d40b1a6becddcb0c853be/roles/370601cf296e44ea95990467138eb5e5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:05.982005 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:05.982330 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:05.982665 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.016048 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] MFA Rules not processed for user `040bc69a909d40b1a6becddcb0c853be`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:06.017888 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:06.089555 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1437/2883] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:06.094284 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.160384 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5343ac-986d-4e7a-889e-a113536d8850 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.165926 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.166656 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.166938 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.167251 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.167936 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.169850 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.198076 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1447/2884] 10.222.0.13 () {60 vars in 1152 bytes} [Thu Jun 30 04:31:06 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 04:31:06.202250 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.213712 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d217d-0321-4f0b-a41f-c73abfcb25c7 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.273507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.274211 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.274507 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.274792 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.275556 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.277678 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.300093 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1438/2885] 10.222.0.13 () {60 vars in 1143 bytes} [Thu Jun 30 04:31:06 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 04:31:06.304479 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.315493 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac2506a-e7f7-43f3-b87e-fcb1258fb9b6 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.326978 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.327999 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.328417 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.328842 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.329612 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.331755 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.349227 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1448/2886] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:31:06.355541 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.369508 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41a4a1-7123-4686-8cbe-16ae78f4ecc2 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.378145 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.379505 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.379816 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.380124 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] PATH_INFO: `/v3/projects/eaa92e196a0e4c55a357020acaa37211/users/5aee263a2e6943579fd91cadcfb39999/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.399808 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa92e196a0e4c55a357020acaa37211, user_id=5aee263a2e6943579fd91cadcfb39999, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.402484 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.404772 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:06.418350 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1439/2887] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:31:06 2022] PUT /identity/v3/projects/eaa92e196a0e4c55a357020acaa37211/users/5aee263a2e6943579fd91cadcfb39999/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:06.422331 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.432361 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa3286-2bb2-4418-abb8-f96e3cec41f7 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.489182 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.489873 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.490165 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.490445 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.491218 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.492845 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd00e-bf63-4fca-b64a-1d5fa2934dbb admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.503075 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1449/2888] 10.222.0.13 () {58 vars in 1121 bytes} [Thu Jun 30 04:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 04:31:06.507353 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.520756 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33359c9-82c9-4455-a3db-b4fb02ba6bf2 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.525966 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.526880 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.527197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.527467 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] PATH_INFO: `/v3/projects/eaa92e196a0e4c55a357020acaa37211/users/5aee263a2e6943579fd91cadcfb39999/roles/7757cc59ef6443ada6dd11c520b3688b` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.529471 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa92e196a0e4c55a357020acaa37211, user_id=5aee263a2e6943579fd91cadcfb39999, role_id=7757cc59ef6443ada6dd11c520b3688b)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.532157 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.543028 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1440/2889] 10.222.0.13 () {60 vars in 1482 bytes} [Thu Jun 30 04:31:06 2022] PUT /identity/v3/projects/eaa92e196a0e4c55a357020acaa37211/users/5aee263a2e6943579fd91cadcfb39999/roles/7757cc59ef6443ada6dd11c520b3688b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:06.553195 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] REQUEST_METHOD: `POST` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.553492 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.553778 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.580793 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.auth.core [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] MFA Rules not processed for user `5aee263a2e6943579fd91cadcfb39999`. Rule list: `[]` (Enabled: `True`). {{(pid=80702) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:06.582019 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:06.591437 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:06.669803 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1450/2890] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:06.681689 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:06.683662 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:06.754038 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8991-c6ff-42cb-bec4-075d5d450c3c admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:06.758375 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:06.759067 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:06.759380 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:06.759660 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:06.824071 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:06.825591 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:06.835139 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1441/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:31:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:08.365107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:08.366543 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:08.379585 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09041ce5-6e12-4e58-af64-eb45e63e4085 None None] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:08.383371 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:08.384012 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:08.384302 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:08.384576 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:08.443647 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:08.445035 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:08.458092 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1451/2892] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:31:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:08.988544 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:08.997747 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13458899-d530-478a-b052-7086b11ef18c service cinder] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:09.001827 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd534d56b90834341afb915c39e70faff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:09.002470 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:09.002763 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:09.003039 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:09.012511 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:09.013950 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114b06e-fb4a-4844-926f-211788765d4e service cinder] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:09.023797 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1442/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:31:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:12.930745 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:13.005447 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6c3a5-4d18-4bbd-a224-353bf86cbb7e service cinder] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:13.010947 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:13.011965 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:13.012256 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:13.012532 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] PATH_INFO: `/v3/users/d916c4f90723401d8e5b0a93ba00ad8c` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:13.013862 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d916c4f90723401d8e5b0a93ba00ad8c)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:13.016416 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:13.072415 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:13.074101 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd916c4f90723401d8e5b0a93ba00ad8c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.087122 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd916c4f90723401d8e5b0a93ba00ad8c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.093847 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd916c4f90723401d8e5b0a93ba00ad8c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.094519 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd916c4f90723401d8e5b0a93ba00ad8c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.095157 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd916c4f90723401d8e5b0a93ba00ad8c'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.108924 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1452/2894] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:12 2022] DELETE /identity/v3/users/d916c4f90723401d8e5b0a93ba00ad8c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:13.113790 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] REQUEST_METHOD: `POST` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:13.114329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:13.114329 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:13.131669 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.auth.core [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] MFA Rules not processed for user `d70cac4e4add4da9a183025769b64cc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80703) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 04:31:13.132891 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:13.202482 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1443/2895] 10.222.0.13 () {58 vars in 957 bytes} [Thu Jun 30 04:31:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 04:31:13.212577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:13.214296 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:13.282153 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8f656-0c4f-465c-81fd-d1213fc443aa admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:13.286717 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00a87fd4c488404f98aa12190f598cfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:13.287407 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:13.287693 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:13.288049 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:13.298179 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:13.299668 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:13.309353 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1453/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:31:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:31:13.598939 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:13.600502 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:13.615321 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e1f9b4-c5f0-4e80-9db1-794461596726 None None] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:13.620442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:13.620835 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:13.621151 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:13.621442 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] PATH_INFO: `/v3/projects/633e27654f1447afa8eba59347024420` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:13.622612 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633e27654f1447afa8eba59347024420)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:13.624540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:13.643938 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:13.684298 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633e27654f1447afa8eba59347024420', 'request_id': 'req-d3621952-17c7-4d7b-9a70-3748018a8579'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.692846 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633e27654f1447afa8eba59347024420', 'request_id': 'req-d3621952-17c7-4d7b-9a70-3748018a8579'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.698583 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633e27654f1447afa8eba59347024420', 'request_id': 'req-d3621952-17c7-4d7b-9a70-3748018a8579'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.710302 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] The token cache is being invalidate because project 633e27654f1447afa8eba59347024420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:13.710598 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.711345 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.712984 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1444/2897] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:13 2022] DELETE /identity/v3/projects/633e27654f1447afa8eba59347024420 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:13.717356 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:13.730292 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a79600-422f-4c0b-9f39-fb202ff78914 service neutron] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:13.794294 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:13.795210 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:13.795491 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:13.795769 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] PATH_INFO: `/v3/users/040bc69a909d40b1a6becddcb0c853be` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:13.797026 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=040bc69a909d40b1a6becddcb0c853be)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:13.798940 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:13.846577 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '040bc69a909d40b1a6becddcb0c853be'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.858161 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '040bc69a909d40b1a6becddcb0c853be'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.860118 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:13.866312 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '040bc69a909d40b1a6becddcb0c853be'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.867226 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '040bc69a909d40b1a6becddcb0c853be'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.868779 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '040bc69a909d40b1a6becddcb0c853be'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:13.882156 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1454/2898] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:13 2022] DELETE /identity/v3/users/040bc69a909d40b1a6becddcb0c853be => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:14.167142 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:14.168735 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:14.182521 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3621952-17c7-4d7b-9a70-3748018a8579 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:14.243540 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:14.244427 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:14.244708 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:14.244978 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] PATH_INFO: `/v3/projects/7feefd21eb33483bb64bf68462b4d8dc` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:14.246209 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7feefd21eb33483bb64bf68462b4d8dc)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:14.248197 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:14.297310 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7feefd21eb33483bb64bf68462b4d8dc', 'request_id': 'req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.305597 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7feefd21eb33483bb64bf68462b4d8dc', 'request_id': 'req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.311175 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7feefd21eb33483bb64bf68462b4d8dc', 'request_id': 'req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.322198 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] The token cache is being invalidate because project 7feefd21eb33483bb64bf68462b4d8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:14.322493 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.323150 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.324724 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1445/2899] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:14 2022] DELETE /identity/v3/projects/7feefd21eb33483bb64bf68462b4d8dc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:14.328897 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:14.330382 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:14.343262 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a6ade-08a9-4097-97c3-6389f9979f7f admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:14.401198 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:14.402083 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:14.402364 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:14.402637 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] PATH_INFO: `/v3/users/5aee263a2e6943579fd91cadcfb39999` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:14.403932 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aee263a2e6943579fd91cadcfb39999)` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:14.405875 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:14.454834 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aee263a2e6943579fd91cadcfb39999'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.467275 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aee263a2e6943579fd91cadcfb39999'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.473994 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aee263a2e6943579fd91cadcfb39999'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.474720 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aee263a2e6943579fd91cadcfb39999'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.475393 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.notifications [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aee263a2e6943579fd91cadcfb39999'} {{(pid=80702) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.490107 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1455/2900] 10.222.0.13 () {58 vars in 1223 bytes} [Thu Jun 30 04:31:14 2022] DELETE /identity/v3/users/5aee263a2e6943579fd91cadcfb39999 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:14.775295 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Authenticating user token {{(pid=80703) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:31:14.776918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.fernet_utils [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80703) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:31:14.790289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bee33-c5b2-41fe-9cfc-926b34e0cb16 admin admin] Validating token access rules against request {{(pid=80703) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:31:14.823324 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:14.849918 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd70cac4e4add4da9a183025769b64cc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '447338b8480048d8be1a3dac45927add', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80703) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:31:14.850806 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:31:14.851089 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:31:14.851393 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] PATH_INFO: `/v3/projects/eaa92e196a0e4c55a357020acaa37211` {{(pid=80703) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:31:14.852593 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa92e196a0e4c55a357020acaa37211)` {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:31:14.854525 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] RBAC: Authorization granted {{(pid=80703) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:31:14.905558 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa92e196a0e4c55a357020acaa37211', 'request_id': 'req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.914289 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa92e196a0e4c55a357020acaa37211', 'request_id': 'req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.920032 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa92e196a0e4c55a357020acaa37211', 'request_id': 'req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2'} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.931870 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] The token cache is being invalidate because project eaa92e196a0e4c55a357020acaa37211 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80703) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 04:31:14.932156 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.932904 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG keystone.notifications [None req-1aecb0f1-7a77-40a1-9c3a-66170ca8b4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80703) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 04:31:14.934588 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: [pid: 80703|app: 0|req: 1446/2901] 10.222.0.13 () {58 vars in 1232 bytes} [Thu Jun 30 04:31:14 2022] DELETE /identity/v3/projects/eaa92e196a0e4c55a357020acaa37211 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 04:31:24.486483 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=58,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:31:24.928726 dv-cinder-846842-7-iscsi devstack@keystone.service[80703]: DEBUG dbcounter [-] [80703] Writing DB stats keystone:SELECT=35,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80703) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 04:33:26.777093 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Authenticating user token {{(pid=80702) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 04:33:26.778635 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:33:26.858052 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b67622-c040-4c24-9494-f9c0fd35ce01 admin admin] Validating token access rules against request {{(pid=80702) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 04:33:26.923784 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe214bb1d7bd4e6abd8256cab5ff579f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f78b240b6174aa8bdcbcdbdc26a36ce', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80702) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 04:33:26.924443 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] REQUEST_METHOD: `GET` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 04:33:26.924723 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] SCRIPT_NAME: `/identity` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 04:33:26.925013 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80702) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 04:33:26.926998 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.fernet_utils [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80702) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 04:33:26.938444 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 04:33:26.940073 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069b08b-c38e-4eee-9a7d-e253af1496cc service placement] RBAC: Authorization granted {{(pid=80702) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 04:33:26.950149 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: [pid: 80702|app: 0|req: 1456/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Thu Jun 30 04:33:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 04:33:36.946192 dv-cinder-846842-7-iscsi devstack@keystone.service[80702]: DEBUG dbcounter [-] [80702] Writing DB stats keystone:SELECT=58 {{(pid=80702) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}